ENH: add support for PEP 803 abi3t with Python 3.15.0b2+#856
Conversation
Add tag/extension support and initial tests for `abi3t` as specified in PEP 803 and implemented in Python 3.15.0b2. This is currently limited to building `abi3t` extensions from freethreading builds of Python, since Meson does not support forcing `abi3t` builds explicitly right now. See mesonbuild/meson#15637 for the relevant discussion. The additional test case utilizes the new API introduced in PEP 793 and PEP 820, and therefore requires Python 3.15.0b2. The relevant test tests wheel correctness both with GIL-enabled and free-threading builds of Python 3.15.0b2. The behavior for older versions of Python remains unchanged. Signed-off-by: Michał Górny <mgorny@quansight.com>
|
I didn't touch the test matrix, since testing on 3.15 is already being addressed by #853. |
Signed-off-by: Michał Górny <mgorny@quansight.com>
Signed-off-by: Michał Górny <mgorny@quansight.com>
|
|
||
|
|
||
| @pytest.mark.skipif(NOGIL_BUILD and CYTHON_VERSION < (3, 1, 0), | ||
| @pytest.mark.skipif(mesonpy._is_free_threaded() and CYTHON_VERSION < (3, 1, 0), |
There was a problem hiding this comment.
I think the NOGIL_BUILD was more self explanatory. Reading the function call mesonpy._is_free_threaded() I would imagine that what has the property of being free threaded is mesonpy, which does not make much sense.
I would keep the old constant.
There was a problem hiding this comment.
Do you think it would be better to put it as a constant in mesonpy? I was torn between using a function and a constant, given this is fixed value.
There was a problem hiding this comment.
Constant seems fine to me. Slight preference for FREE_THREADED_BUILD over NOGIL_BUILD (the latter name will fade over time I think), but I don't care that much either way.
There was a problem hiding this comment.
@rgommers IIRC it was you that introduced the NOGIL_BUILD constant 🙂
There was a problem hiding this comment.
Guilty:) I guess that was early days, before the Steering Council pronounced that that was a negative name and they wanted everyone to use free-threading/free-threaded everywhere.
There was a problem hiding this comment.
For now, I've focused on the changes relevant to the PR. Do you want me to change the constant name, and possibly move it to mesonpy/__init__.py here, or separately?
There was a problem hiding this comment.
For now, I've focused on the changes relevant to the PR.
That sounds fine to me - small diffs are nice
Co-authored-by: Daniele Nicolodi <daniele@grinta.net>
This reverts commit a23fe90. Signed-off-by: Michał Górny <mgorny@quansight.com>
Signed-off-by: Michał Górny <mgorny@quansight.com>
Signed-off-by: Michał Górny <mgorny@quansight.com>
Signed-off-by: Michał Górny <mgorny@quansight.com>
|
Downstream testing, and releasing after rc1 in early August, is currently blocked on this, so I added a v0.21.0 tag - we should do that next month I think. |
This reverts commit 0d43c65. Signed-off-by: Michał Górny <mgorny@quansight.com>
Signed-off-by: Michał Górny <mgorny@quansight.com>
rgommers
left a comment
There was a problem hiding this comment.
A couple of small issues in the test extension. The rest looks right; I refreshed my memory on Py_TARGET_ABI3T and that should be added automatically in Python.h for a free-threaded interpreter if Py_LIMITED_API is defined - so all good there.
| PyABIInfo_VAR(abi_info); | ||
|
|
||
| static PySlot module_slots[] = { | ||
| PySlot_STATIC_DATA(Py_mod_name, "plat"), |
There was a problem hiding this comment.
"plat" should be "module" right, to match the filename and the PyModExport_module below?
There was a problem hiding this comment.
To be honest, I've copied the name from the limited-api test. I have no clue why it's "plat" there, so I've assumed there must be a reason :-).
There was a problem hiding this comment.
Changed. I suspect I'm going to follow this up with "misc. fixes" PR xD.
There was a problem hiding this comment.
You changed it in the old test, but not here yet
There was a problem hiding this comment.
Uh, sorry about that. Should be correct now.
| if '__pypy__' in sys.builtin_module_names: | ||
| abi = ABI | ||
| elif NOGIL_BUILD and sys.version_info >= (3, 15): | ||
| abi = 'abi3.abi3t' |
There was a problem hiding this comment.
I think that, if this works, it does by chance. The fake data passed to the wheel builder uses ABI3SUFFIX which is defined at the top of this module. That that resolves to abi3t because of the orders in which the suffixes are listed in importlib.machinery.EXTENSION_SUFFIXES. Is the order guaranteed or should we try to make this more robust? If the order is guaranteed, I would still add a comment in the get_abi3_suffix() explaining what it is expected and why it works.
There was a problem hiding this comment.
A comment would be great. My understanding is that it is guaranteed - and Meson relies on it as well, which came up in the PEP 803 discussions. (that said, future-proofing might be wise).
There was a problem hiding this comment.
Yes, it is guaranteed, CPython is driving module loading on it. For GIL-enabled, it's:
>>> importlib.machinery.EXTENSION_SUFFIXES
['.cpython-315-x86_64-linux-gnu.so', '.abi3.so', '.abi3t.so', '.so']It's ordered from most specific (i.e. presumably most performant) to least specific, to .so legacy fallback. I'll add a comment.
There was a problem hiding this comment.
Doesn't this mean that get_abi3_suffix() on Python 3.15t returns .abi3.so and this test fails because when building on Python 3.15t the extension modules should be suffixed with .abi3t.so?
There was a problem hiding this comment.
Doesn't this mean that get_abi3_suffix() on Python 3.15t returns .abi3.so and this test fails because when building on Python 3.15t the extension modules should be suffixed with .abi3t.so?
Nope, EXTENSION_SUFFIXES is different on 3.15t:
Python 3.15.0b2 free-threading build (main, Jun 4 2026, 14:23:26) [Clang 21.0.0 (clang-2100.1.1.101)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib.machinery
>>> importlib.machinery.EXTENSION_SUFFIXES
['.cpython-315t-darwin.so', '.abi3t.so', '.so']
This is actually explicitly specified in PEP 803: https://peps.python.org/pep-0803/#the-abi3t-wheel-and-filename-tags
There was a problem hiding this comment.
And yes, abi3.so is included on 3.14t and 3.13t and that's probably a bug but it was only noticed in the discussion around PEP 803 after 3.14t came out. Arguably we probably should have fixed this in the lead-up to 3.13t but no one noticed. I have no idea if removing .abi3.so from EXTENSION_SUFFIXES on 3.14t would have fallout.
Signed-off-by: Michał Górny <mgorny@quansight.com>
Signed-off-by: Michał Górny <mgorny@quansight.com>
Signed-off-by: Michał Górny <mgorny@quansight.com>
Signed-off-by: Michał Górny <mgorny@quansight.com>
|
This is looking pretty good now. Are you expecting it to be squash-merged, or do you want to rewrite history? (we usually do the latter, and rebase-merge) |
Add tag/extension support and initial tests for
abi3tas specified in PEP 803 and implemented in Python 3.15.0b2. This is currently limited to buildingabi3textensions from freethreading builds of Python, since Meson does not support forcingabi3tbuilds explicitly right now. See mesonbuild/meson#15637 for the relevant discussion.The additional test case utilizes the new API introduced in PEP 793 and PEP 820, and therefore requires Python 3.15.0b2. The relevant test tests wheel correctness both with GIL-enabled and free-threading builds of Python 3.15.0b2.
The behavior for older versions of Python remains unchanged.