If so, I'm also wondering if I could strip the thumbnail parent div out of the backref files so that I could do it in the loop.

Not sure if this will break anything or not. But you could do it and we could check to see if things still render properly.

And if it makes sense to add a ... arg that will look in the folder for a readme that defines the sort order?

I'm not sure how it would work to use RST to define the order, but agree it makes sense to define it somewhere somehow.

A solution that would be more consistent with other order setting mechanics would be to provide a sphinx_gallery_conf["minigallery_sort_order"] that's a callable that takes whatever it needs to take (probably the current source file and list of examples or something?) and returns the sorted order or so.

I'm not sure how it would work to use RST to define the order, but agree it makes sense to define it somewhere somehow.

Was thinking about this 'cause #1068 but realizing the answer to both this is probably a FromFileOrder type object. ETA or like Tim's suggestion, expand the matplotlib ordering object to also look in files.

b/c I have gone down way too deep an html/xml rabbit hole, does it matter if the heading is before or in the thumbnail parent div? my bias is for in b/c then it stays together as part of the DOM

ETA: also main reason for completely rewriting the minigallery test was to get a better sense of expected behavior.

I have gone down way too deep an html/xml rabbit hole

I have nothing to contribute here but I just wanted to thank you for your hard work.

Also the CIs are failing because scipy docs site certificate has expired scipy/docs.scipy.org#80

does it matter if the heading is before or in the thumbnail parent div? my bias is for in b/c then it stays together as part of the DOM

I am not really sure but if I had to guess I would say no it shouldn't matter, so feel free to try moving it

. so feel free to try moving it

Moved it inside in this PR and the build looks the same to me.

that's a callable that takes whatever it needs to take (probably the current source file and list of examples or something?) and returns the sorted order or so.

I think a standalone/separate PR would be to make something like a FileOrder(ExplicitOrder) object that can be configured with the file it should look for the ordering, and that returns a callable that works like ExplicitOrder and that for this PR I should just make sure that a [minigallery] ordering config of some sort will work.

Also test failures are I think cause I moved the title into the div/changed the backreferences file, so I'll sort that out probably in the coming week or so.

I think a standalone/separate PR would be to make something like a FileOrder(ExplicitOrder) object that can be configured with the file it should look for the ordering

Thinking about this more... I'm not sure we can/should implement this at the SG end, at least to start. You could already implement and use this in matplotlib because you can define your own sortkey if you want. For example following this code:

https://sphinx-gallery.github.io/stable/_modules/sphinx_gallery/sorting.html#ExampleTitleSortKey

I'd expect something like the following, where the ... is whatever you need to do to parse the RST in whatever way matplotlib wants to specify the information:

class RSTSortKey:

    def __init__(self, src_dir):
        self.src_dir = src_dir
        readme_txt = Path(src_dir) / "README.rst").read_text("utf-8")
        self.fname_list_orderered = ...  # something you pull from the readme

    def __call__(self, filename):
        return self.fname_list_ordered.index(filename)

... in other words, I think the hard/interesting work here is in the ... above, and that's going to be a bit specific to how matplotlib does things. But assuming that works, it would be great to link from the SG config docs to the matplotlib code showing how the SortKey-based framework is general enough to support this use case!

This is amazing and also probably a sign I messed up divs. ETA: confirmed, this is what happens if I try to put the heading inside the thumbnail div 😅

So pushed into a separate commit in case you want to reject, because I understand why you'd want to frontload the FunctionSortKey but I pulled the repr discussion up 'cause I thought the section flow was jumpy otherwise.

That's fine, I've used this ordering.

Some minor changes:

• I've just used .name to get the filename instead of the full path (it's not pretty but we were manipulating the sort elements anyway so 🤷) - did not find a way to test this directly but I have amended MockSort so it will fail if not just the filename
• I've allowed None as a minigallery_sort_order input - could not think of way to test this directly, though i noticed we don't test this for subsection_order either
• I don't think the doc config minigallery_sort_order in the SG docs was working (as we were passing the full path before) and I didn't think it was necessary. Also now something fails if minigallery_sort_order does not allow None
• Doc amendments. I have realised that you cannot use FunctionSortKey for within_subsection_order config as they get instantiated with the source dir

Thanks @story645 for all your work. Just made minor fixes to save you time.

I've just used .name to get the filename instead of the full path (it's not pretty but we were manipulating the sort elements anyway so 🤷) - did not find a way to test this directly but I have amended MockSort so it will fail if not just the filename

Mentioned this in a review, but this won't let you keep files in a subfolder together and will collapse two files of the same name in different subfolders & I think those are some of the primary reasons I'd want to sort.

Ah sorry I missed that. What about the making the paths relative to src_dir ?

We could probably do that here:

            elif paths := Path(src_dir).glob(obj):
                file_paths.extend([(obj, p) for p in paths])

?

Maybe we could add a test to capture this? And update the documentation so people know what we're sorting on.

Let me know if you don't want to work on it.

Let me know if you don't want to work on it.

I was actually gonna open up a PR to put back sorting on the full path.

What making the making the paths relative to src_dir?

    elif paths := Path(src_dir).glob(obj):

            file_paths.extend([(obj, p) for p in paths])

Path(src_dir).glob(obj) returns a Path relative to source dir, while os.path.abspath and .absolute get rid of it, and I think maybe the solution is put that in the sortkey -> it can't be done here b/c the source parsing files all want the relative to src_dir path far as I can tell. The issue is that the examples folders are not subfolders of src_dir so trying to make them absolute loses the relative position to src_dir.

That's fine. We can do it in the sortkey, maybe adding a parent ? I think we only ever go 1 folder deep.

I think we only ever go 1 folder deep.

Ehh, Matplotlib does some wonky nested nonsense so I'd rather be as general as possible.

Sure I'm okay with it the previous status, but it should be documented so people know what we're sorting on (and that it's not just the filename). And tested if possible...

Actually, the filepath input arg to the directive would be okay? Since they have to give the filepath relative to conf.py so subfolder structure would be retained?

Actually, the filepath input arg to the directive would be okay?

Don't think so b/c it can also be a glob expression ../*/*.py

Oh yeah good point.

I realise now why I thought it should be the filename, the example we give is along the lines of x.startswith('plot'). That would be trickier with the full path, they would have split the path but that's unavoidable if we want the subdir in there.

Ehh, Matplotlib does some wonky nested nonsense so I'd rather be as general as possible.

I meant that I think sphinx gallery only looks 1 level down for subdirs, though matplotlib may do funny things to get around this?

I meant that I think sphinx gallery only looks 1 level down for subdirs, though matplotlib may do funny things to get around this?

Hmm, I'm probably wrong then and we only have one level of subdirs? Where https://github.com/matplotlib/matplotlib/tree/main/galleries is our tree...hmm,

they would have split the path but that's unavoidable if we want the subdir in there.

os.path.basename(x).startswith('plot')

I think a worthwhile followup PR is for the sortkey callables to take Path or string, but is out of scope here I think.