Thanks @h-vetinari, I wonder if we should add a note somewhere that users who compile with the stable Python API for a range of versions should customize this to ensure things work if NumPy publishes a new version.

I wonder if we should add a note somewhere that users who compile with the stable Python API for a range of versions should customize this to ensure things work if NumPy publishes a new version.

The consistent message I got from @rgommers on this was "don't touch it; in fact, don't even tell anyone about NPY_FEATURE_VERSION" (e.g. here). My understanding is that the constraint here is both maximally-compatible as well as self-enforcing, in the sense that the lower bound for Python (3.10) means that targetting anything before 1.21 is pointless (because earlier numpy's didn't support 3.10).

Also, this should IMO be backported to 2.1.0; CC @charris

Ah sorry, saw too late that you had already added the label - apologies!

My understanding is that the constraint here is both maximally-compatible as well as self-enforcing

Maybe the advice can be to keep using the same Python version for bug-fix releases, dunno.
Otherwise, if a package decides to recompile with the newest Python version always (which a stable API package can do).
They would implicitly drop support for old Python versions (because they have to use the lastest NumPy which then fails to support the old Python).

So such packages would need to take some care even if few (if any) exist right now that actually use the stable API.

I suspect there is actually already a mention somewhere that a stable API user may wish to use it to support a wider range of Python versions than NumPy itself, though.

I suspect that such packages don't exist. I experimented a bit with using the stable ABI, but there's very little you can do with both Cython and NumPy while staying within the limited C API.

If we're going to worry about this, we really should have a test package that does something more than a no-op import of headers (as in _core/tests/examples/limited_api/).

They would implicitly drop support for old Python versions (because they have to use the lastest NumPy which then fails to support the old Python).

I'm also not sure this is correct. This change drops support for numpy<1.21, but not for Python 3.9.

This change drops support for numpy<1.21, but not for Python 3.9.

I mean, that's kind of vacuously true because support for 3.9 has already been dropped? Was that what you wanted to say?

So far, I don't understand the use case @seberg is describing. No one has to use the latest numpy - people got along fine for years with oldest-supported-numpy. Nowadays they have the option to build with newer NumPy without losing compatibility with older versions, but if they do insist on the latest one, they're limited to python >=3.10 (and thus implicitly numpy >=1.21).

This change drops support for numpy<1.21, but not for Python 3.9.

Ah, right. Which could still be unintentional, but much less impactful in practice.

I was merely wondering if should think more of stable API users for whom the default cannot be maximally compatible...

Ah, sorry, I got the stable API part too late. Isn't there still an implicit python-version-dependence when one builds against the stable API/ABI? I.e. newer python versions support a larger surface of features for the stable ABI, so something compiled against the default 3.12 stable ABI is not necessarily usable with older pythons.

From what I can tell, the situation is similar to numpy 2.0, in that newer pythons allow targeting older stable ABIs. For example, cryptography builds abi3 wheels for py37 & py39 on osx, c.f. pyca/cryptography@a9a4f5d

I was merely wondering if should think more of stable API users for whom the default cannot be maximally compatible...

I think we could cover this with a single sentence somewhere, like: "if you are targeting the CPython Stable ABI and want to support very old NumPy versions and are not building with the lowest Python version you are targeting, then please ensure to use a numpy version at build time that has support for that lowest Python version".

But again, this only makes sense to add if we actually add docs on Stable ABI support and usage. Right now we don't have anything, so this sentence by itself is only going to confuse people.

Isn't there still an implicit python-version-dependence when one builds against the stable API/ABI? I.e. newer python versions support a larger surface of features for the stable ABI, so something compiled against the default 3.12 stable ABI is not necessarily usable with older pythons.

No you can do this, say build on Python 3.12 but target Python >=3.9.

No you can do this, say build on Python 3.12 but target Python >=3.9.

This is exactly what I wrote in the second paragraph... (and why the first one used "default 3.12 stable ABI").

I don't really care. I do think people will run into this eventually, but they'll notice. pyerfa is already in that position, I think if they were to do a bug-fix release. And we had one other project opening a compile time warning with the limited API (compiling with newer Python versions). Also Cython supports it, so I would think others will try. And hopefully anyone considering it will correctly assume that Python limited API use is orthogonal to NumPy C-API use (even if there are two minuscule caveats and maybe still a niche stray bad macro).