Feature or enhancement

Proposal:

In free-threaded builds, the specializing adaptive interpreter needs to be made thread-safe. We should start with a small PR to simply disable it in free-threaded builds, which will be correct but will incur a performance penalty. Then we can work out how to properly support specialization in a free-threaded build.

These two commits from Sam's nogil-3.12 branch can serve as inspiration:

1. specialize: make specialization thread-safe
   
2. specialize: optimize for single-threaded programs

There are two primary concerns to balance while implementing this functionality on main:

1. Runtime overhead: There should be no performance impact on normal builds, and minimal performance impact on single-threaded code running in free-threaded builds.
2. Reducing code duplication/divergence: We should come up with a design that is minimally disruptive to ongoing work on the specializing interpreter. It should be easy for other devs to keep the free-threaded build working without having to know too much about it.

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

• https://peps.python.org/pep-0703/
• PEP 703 -- Making the Global Interpreter Lock Optional in CPython #108219

Specialization Families

- [x] BINARY_OP
- [x] BINARY_SUBSCR - @corona10
- [x] CALL - @mpage
- [x] CALL_KW - @mpage
- [x] COMPARE_OP - @Yhg1s
- [x] CONTAINS_OP - @corona10
- [x] FOR_ITER - @Yhg1s
- [x] LOAD_ATTR - @mpage
- [x] LOAD_CONST
- [x] LOAD_GLOBAL - @mpage
- [x] LOAD_SUPER_ATTR - @nascheme
- [x] RESUME
- [x] SEND - @nascheme
- [x] STORE_ATTR - -@nascheme
- [x] STORE_SUBSCR - @colesbury
- [x] TO_BOOL - @corona10
- [x] UNPACK_SEQUENCE - @Eclips4

Linked PRs

• gh-115999: Disable the specializing adaptive interpreter in free-threaded builds #116013
• gh-115999: Implement thread-local bytecode and enable specialization for BINARY_OP #123926
• gh-115999: Refactor LOAD_GLOBAL specializations to avoid reloading {globals, builtins} keys #124953
• gh-115999: Stop the world when invalidating function versions #124997
• gh-115999: Add free-threaded specialization for COMPARE_OP #126410
• gh-115999: Introduce helpers for (un)specializing instructions #126414
• gh-115999: Fix gdb support for libpython.so after thread-local bytecode change #126440
• gh-115999: Add free-threaded specialization for CONTAINS_OP #126450
• gh-115999: Move specializer test from test_dis to test_opcache #126498
• gh-115999: Restore SET_OPCODE_OR_RETURN removed in GH-126414 #126515
• gh-115999: Add free-threaded specialization for UNPACK_SEQUENCE #126600
• gh-115999: Specialize LOAD_GLOBAL in free-threaded builds #126607
• gh-115999: Add free-threaded specialization for TO_BOOL #126616
• gh-115999: Don't take a reason in unspecialize #127030
• gh-115999: Enable specialization of CALL instructions in free-threaded builds #127123
• gh-115999: Specialize LOAD_SUPER_ATTR in free-threaded builds #127128
• gh-115999: Record success in specialize #127167
• gh-115999: Add free-threaded specialization for STORE_SUBSCR #127169
• gh-115999: Add partial free-thread specialization for BINARY_SUBSCR #127227
• gh-115999: Add free-threaded specialization for SEND. #127426
• gh-115999: Use light-weight lock for UNPACK_SEQUENCE_LIST #127514
• gh-115999: Specialize loading attributes from modules in free-threaded builds #127711
• gh-115999: Specialize CALL_KW in free-threaded builds #127713
• gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build #127737
• gh-115999: Specialize STORE_ATTR in free-threaded builds. #127838
• gh-115999: Specialize LOAD_ATTR for instance and class receivers in free-threaded builds #128164
• gh-115999: Update test_opcache to test with nested method #128166
• gh-115999: Make list and tuple iteration more thread-safe. #128637
• gh-115999: Add free-threaded specialization for FOR_ITER #128798
• gh-115999: Enable free-threaded specialization of LOAD_CONST #129365
• gh-115999: Note Python 3.14 free-threaded changes in What's New #131285
• [3.14] gh-115999: Note Python 3.14 free-threaded changes in What's New (gh-131285) #133824
• gh-115999: Add PyCodeObject.co_tlbc to the debug offsets #134286
• [3.14] gh-115999: Add PyCodeObject.co_tlbc to the debug offsets (GH-134286) #134348
• gh-115999: remove redundant check in free-threading from _STORE_ATTR_WITH_HINT #136249