Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 86d5fa9

Browse filesBrowse files
ZeroIntensityvstinner
ZeroIntensity
and
vstinner
authored
gh-127989: C API: Refer to "attached thread states" instead of the GIL (GH-127990)
Co-authored-by: Victor Stinner <vstinner@python.org>
1 parent 83d54fa commit 86d5fa9
Copy full SHA for 86d5fa9

13 files changed

+250
-190
lines changed

‎Doc/c-api/dict.rst

Copy file name to clipboardExpand all lines: Doc/c-api/dict.rst
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ Dictionary Objects
127127
Prefer the :c:func:`PyDict_GetItemWithError` function instead.
128128
129129
.. versionchanged:: 3.10
130-
Calling this API without :term:`GIL` held had been allowed for historical
130+
Calling this API without an :term:`attached thread state` had been allowed for historical
131131
reason. It is no longer allowed.
132132
133133

‎Doc/c-api/exceptions.rst

Copy file name to clipboardExpand all lines: Doc/c-api/exceptions.rst
+3-3Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -413,7 +413,7 @@ Querying the error indicator
413413
own a reference to the return value, so you do not need to :c:func:`Py_DECREF`
414414
it.
415415
416-
The caller must hold the GIL.
416+
The caller must have an :term:`attached thread state`.
417417
418418
.. note::
419419
@@ -675,7 +675,7 @@ Signal Handling
675675
676676
.. note::
677677
This function is async-signal-safe. It can be called without
678-
the :term:`GIL` and from a C signal handler.
678+
an :term:`attached thread state` and from a C signal handler.
679679
680680
681681
.. c:function:: int PyErr_SetInterruptEx(int signum)
@@ -702,7 +702,7 @@ Signal Handling
702702
703703
.. note::
704704
This function is async-signal-safe. It can be called without
705-
the :term:`GIL` and from a C signal handler.
705+
an :term:`attached thread state` and from a C signal handler.
706706
707707
.. versionadded:: 3.10
708708

‎Doc/c-api/init.rst

Copy file name to clipboardExpand all lines: Doc/c-api/init.rst
+157-144Lines changed: 157 additions & 144 deletions
Large diffs are not rendered by default.

‎Doc/c-api/init_config.rst

Copy file name to clipboardExpand all lines: Doc/c-api/init_config.rst
+6-6Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -578,8 +578,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option
578578
* ``list[str]``
579579
* ``dict[str, str]``
580580
581-
The caller must hold the GIL. The function cannot be called before
582-
Python initialization nor after Python finalization.
581+
The caller must have an :term:`attached thread state`. The function cannot
582+
be called before Python initialization nor after Python finalization.
583583
584584
.. versionadded:: 3.14
585585
@@ -601,8 +601,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option
601601
* Return a new reference on success.
602602
* Set an exception and return ``NULL`` on error.
603603
604-
The caller must hold the GIL. The function cannot be called before
605-
Python initialization nor after Python finalization.
604+
The caller must have an :term:`attached thread state`. The function cannot
605+
be called before Python initialization nor after Python finalization.
606606
607607
.. versionadded:: 3.14
608608
@@ -616,8 +616,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option
616616
* Raise a :exc:`ValueError` if the option is read-only (cannot be set).
617617
* Raise a :exc:`TypeError` if *value* has not the proper type.
618618
619-
The caller must hold the GIL. The function cannot be called before
620-
Python initialization nor after Python finalization.
619+
The caller must have an :term:`attached thread state`. The function cannot
620+
be called before Python initialization nor after Python finalization.
621621
622622
.. versionadded:: 3.14
623623

‎Doc/c-api/memory.rst

Copy file name to clipboardExpand all lines: Doc/c-api/memory.rst
+15-17Lines changed: 15 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -110,12 +110,12 @@ The three allocation domains are:
110110

111111
* Raw domain: intended for allocating memory for general-purpose memory
112112
buffers where the allocation *must* go to the system allocator or where the
113-
allocator can operate without the :term:`GIL`. The memory is requested directly
114-
from the system. See :ref:`Raw Memory Interface <raw-memoryinterface>`.
113+
allocator can operate without an :term:`attached thread state`. The memory
114+
is requested directly from the system. See :ref:`Raw Memory Interface <raw-memoryinterface>`.
115115

116116
* "Mem" domain: intended for allocating memory for Python buffers and
117117
general-purpose memory buffers where the allocation must be performed with
118-
the :term:`GIL` held. The memory is taken from the Python private heap.
118+
an :term:`attached thread state`. The memory is taken from the Python private heap.
119119
See :ref:`Memory Interface <memoryinterface>`.
120120

121121
* Object domain: intended for allocating memory for Python objects. The
@@ -139,8 +139,8 @@ Raw Memory Interface
139139
====================
140140

141141
The following function sets are wrappers to the system allocator. These
142-
functions are thread-safe, the :term:`GIL <global interpreter lock>` does not
143-
need to be held.
142+
functions are thread-safe, so a :term:`thread state` does not
143+
need to be :term:`attached <attached thread state>`.
144144

145145
The :ref:`default raw memory allocator <default-memory-allocators>` uses
146146
the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc`
@@ -213,8 +213,7 @@ The :ref:`default memory allocator <default-memory-allocators>` uses the
213213
214214
.. warning::
215215
216-
The :term:`GIL <global interpreter lock>` must be held when using these
217-
functions.
216+
There must be an :term:`attached thread state` when using these functions.
218217
219218
.. versionchanged:: 3.6
220219
@@ -327,8 +326,7 @@ The :ref:`default object allocator <default-memory-allocators>` uses the
327326
328327
.. warning::
329328
330-
The :term:`GIL <global interpreter lock>` must be held when using these
331-
functions.
329+
There must be an :term:`attached thread state` when using these functions.
332330
333331
.. c:function:: void* PyObject_Malloc(size_t n)
334332
@@ -485,12 +483,12 @@ Customize Memory Allocators
485483
zero bytes.
486484
487485
For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be
488-
thread-safe: the :term:`GIL <global interpreter lock>` is not held when the
489-
allocator is called.
486+
thread-safe: a :term:`thread state` is not :term:`attached <attached thread state>`
487+
when the allocator is called.
490488
491489
For the remaining domains, the allocator must also be thread-safe:
492490
the allocator may be called in different interpreters that do not
493-
share a ``GIL``.
491+
share a :term:`GIL`.
494492
495493
If the new allocator is not a hook (does not call the previous allocator),
496494
the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the
@@ -507,8 +505,8 @@ Customize Memory Allocators
507505
:c:func:`Py_InitializeFromConfig` to install a custom memory
508506
allocator. There are no restrictions over the installed allocator
509507
other than the ones imposed by the domain (for instance, the Raw
510-
Domain allows the allocator to be called without the GIL held). See
511-
:ref:`the section on allocator domains <allocator-domains>` for more
508+
Domain allows the allocator to be called without an :term:`attached thread state`).
509+
See :ref:`the section on allocator domains <allocator-domains>` for more
512510
information.
513511
514512
* If called after Python has finish initializing (after
@@ -555,7 +553,7 @@ Runtime checks:
555553
called on a memory block allocated by :c:func:`PyMem_Malloc`.
556554
- Detect write before the start of the buffer (buffer underflow).
557555
- Detect write after the end of the buffer (buffer overflow).
558-
- Check that the :term:`GIL <global interpreter lock>` is held when
556+
- Check that there is an :term:`attached thread state` when
559557
allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex:
560558
:c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex:
561559
:c:func:`PyMem_Malloc`) domains are called.
@@ -620,8 +618,8 @@ PYMEM_CLEANBYTE (meaning uninitialized memory is getting used).
620618
The :c:func:`PyMem_SetupDebugHooks` function now also works on Python
621619
compiled in release mode. On error, the debug hooks now use
622620
:mod:`tracemalloc` to get the traceback where a memory block was allocated.
623-
The debug hooks now also check if the GIL is held when functions of
624-
:c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are
621+
The debug hooks now also check if there is an :term:`attached thread state` when
622+
functions of :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are
625623
called.
626624
627625
.. versionchanged:: 3.8

‎Doc/c-api/module.rst

Copy file name to clipboardExpand all lines: Doc/c-api/module.rst
+2-2Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -709,7 +709,7 @@ since multiple such modules can be created from a single definition.
709709
mechanisms (either by calling it directly, or by referring to its
710710
implementation for details of the required state updates).
711711
712-
The caller must hold the GIL.
712+
The caller must have an :term:`attached thread state`.
713713
714714
Return ``-1`` with an exception set on error, ``0`` on success.
715715
@@ -720,6 +720,6 @@ since multiple such modules can be created from a single definition.
720720
Removes the module object created from *def* from the interpreter state.
721721
Return ``-1`` with an exception set on error, ``0`` on success.
722722
723-
The caller must hold the GIL.
723+
The caller must have an :term:`attached thread state`.
724724
725725
.. versionadded:: 3.3

‎Doc/c-api/perfmaps.rst

Copy file name to clipboardExpand all lines: Doc/c-api/perfmaps.rst
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ kernel/git/torvalds/linux.git/tree/tools/perf/Documentation/jit-interface.txt>`_
1616
In Python, these helper APIs can be used by libraries and features that rely
1717
on generating machine code on the fly.
1818

19-
Note that holding the Global Interpreter Lock (GIL) is not required for these APIs.
19+
Note that holding an :term:`attached thread state` is not required for these APIs.
2020

2121
.. c:function:: int PyUnstable_PerfMapState_Init(void)
2222

‎Doc/c-api/reflection.rst

Copy file name to clipboardExpand all lines: Doc/c-api/reflection.rst
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ Reflection
5555
5656
.. c:function:: PyFrameObject* PyEval_GetFrame(void)
5757
58-
Return the current thread state's frame, which is ``NULL`` if no frame is
58+
Return the :term:`attached thread state`'s frame, which is ``NULL`` if no frame is
5959
currently executing.
6060
6161
See also :c:func:`PyThreadState_GetFrame`.

‎Doc/c-api/sys.rst

Copy file name to clipboardExpand all lines: Doc/c-api/sys.rst
+3-3Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -232,7 +232,7 @@ Operating System Utilities
232232
233233
The file descriptor is created non-inheritable (:pep:`446`).
234234
235-
The caller must hold the GIL.
235+
The caller must have an :term:`attached thread state`.
236236
237237
.. versionadded:: 3.14
238238
@@ -378,8 +378,8 @@ accessible to C code. They all work with the current interpreter thread's
378378
silently abort the operation by raising an error subclassed from
379379
:class:`Exception` (other errors will not be silenced).
380380
381-
The hook function is always called with the GIL held by the Python
382-
interpreter that raised the event.
381+
The hook function is always called with an :term:`attached thread state` by
382+
the Python interpreter that raised the event.
383383
384384
See :pep:`578` for a detailed description of auditing. Functions in the
385385
runtime and standard library that raise events are listed in the

‎Doc/c-api/time.rst

Copy file name to clipboardExpand all lines: Doc/c-api/time.rst
+7-7Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ range.
5656
system time.)
5757

5858
As any other C API (unless otherwise specified), the functions must be called
59-
with the :term:`GIL` held.
59+
with an :term:`attached thread state`.
6060

6161
.. c:function:: int PyTime_Monotonic(PyTime_t *result)
6262
@@ -78,29 +78,29 @@ Raw Clock Functions
7878
-------------------
7979
8080
Similar to clock functions, but don't set an exception on error and don't
81-
require the caller to hold the GIL.
81+
require the caller to have an :term:`attached thread state`.
8282
8383
On success, the functions return ``0``.
8484
8585
On failure, they set ``*result`` to ``0`` and return ``-1``, *without* setting
86-
an exception. To get the cause of the error, acquire the GIL and call the
87-
regular (non-``Raw``) function. Note that the regular function may succeed after
86+
an exception. To get the cause of the error, :term:`attach <attached thread state>` a :term:`thread state`,
87+
and call the regular (non-``Raw``) function. Note that the regular function may succeed after
8888
the ``Raw`` one failed.
8989
9090
.. c:function:: int PyTime_MonotonicRaw(PyTime_t *result)
9191
9292
Similar to :c:func:`PyTime_Monotonic`,
93-
but don't set an exception on error and don't require holding the GIL.
93+
but don't set an exception on error and don't require an :term:`attached thread state`.
9494
9595
.. c:function:: int PyTime_PerfCounterRaw(PyTime_t *result)
9696
9797
Similar to :c:func:`PyTime_PerfCounter`,
98-
but don't set an exception on error and don't require holding the GIL.
98+
but don't set an exception on error and don't require an :term:`attached thread state`.
9999
100100
.. c:function:: int PyTime_TimeRaw(PyTime_t *result)
101101
102102
Similar to :c:func:`PyTime_Time`,
103-
but don't set an exception on error and don't require holding the GIL.
103+
but don't set an exception on error and don't require an :term:`attached thread state`.
104104
105105
106106
Conversion functions

‎Doc/c-api/typeobj.rst

Copy file name to clipboardExpand all lines: Doc/c-api/typeobj.rst
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -731,7 +731,7 @@ and :c:data:`PyType_Type` effectively act as defaults.)
731731
object becomes part of a refcount cycle, that cycle might be collected by
732732
a garbage collection on any thread). This is not a problem for Python
733733
API calls, since the thread on which :c:member:`!tp_dealloc` is called
734-
will own the Global Interpreter Lock (GIL). However, if the object being
734+
with an :term:`attached thread state`. However, if the object being
735735
destroyed in turn destroys objects from some other C or C++ library, care
736736
should be taken to ensure that destroying those objects on the thread
737737
which called :c:member:`!tp_dealloc` will not violate any assumptions of

‎Doc/extending/newtypes_tutorial.rst

Copy file name to clipboardExpand all lines: Doc/extending/newtypes_tutorial.rst
+4-4Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -403,8 +403,8 @@ the new attribute values. We might be tempted, for example to assign the
403403
But this would be risky. Our type doesn't restrict the type of the
404404
``first`` member, so it could be any kind of object. It could have a
405405
destructor that causes code to be executed that tries to access the
406-
``first`` member; or that destructor could release the
407-
:term:`Global interpreter Lock <GIL>` and let arbitrary code run in other
406+
``first`` member; or that destructor could detach the
407+
:term:`thread state <attached thread state>` and let arbitrary code run in other
408408
threads that accesses and modifies our object.
409409

410410
To be paranoid and protect ourselves against this possibility, we almost
@@ -413,8 +413,8 @@ don't we have to do this?
413413

414414
* when we absolutely know that the reference count is greater than 1;
415415

416-
* when we know that deallocation of the object [#]_ will neither release
417-
the :term:`GIL` nor cause any calls back into our type's code;
416+
* when we know that deallocation of the object [#]_ will neither detach
417+
the :term:`thread state <attached thread state>` nor cause any calls back into our type's code;
418418

419419
* when decrementing a reference count in a :c:member:`~PyTypeObject.tp_dealloc`
420420
handler on a type which doesn't support cyclic garbage collection [#]_.

‎Doc/glossary.rst

Copy file name to clipboardExpand all lines: Doc/glossary.rst
+49Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,28 @@ Glossary
132132
iterator's :meth:`~object.__anext__` method until it raises a
133133
:exc:`StopAsyncIteration` exception. Introduced by :pep:`492`.
134134

135+
attached thread state
136+
137+
A :term:`thread state` that is active for the current OS thread.
138+
139+
When a :term:`thread state` is attached, the OS thread has
140+
access to the full Python C API and can safely invoke the
141+
bytecode interpreter.
142+
143+
Unless a function explicitly notes otherwise, attempting to call
144+
the C API without an attached thread state will result in a fatal
145+
error or undefined behavior. A thread state can be attached and detached
146+
explicitly by the user through the C API, or implicitly by the runtime,
147+
including during blocking C calls and by the bytecode interpreter in between
148+
calls.
149+
150+
On most builds of Python, having an attached thread state implies that the
151+
caller holds the :term:`GIL` for the current interpreter, so only
152+
one OS thread can have an attached thread state at a given moment. In
153+
:term:`free-threaded <free threading>` builds of Python, threads can concurrently
154+
hold an attached thread state, allowing for true parallelism of the bytecode
155+
interpreter.
156+
135157
attribute
136158
A value associated with an object which is usually referenced by name
137159
using dotted expressions.
@@ -622,6 +644,10 @@ Glossary
622644
multi-threaded applications and makes it easier to use multi-core CPUs
623645
efficiently. For more details, see :pep:`703`.
624646

647+
In prior versions of Python's C API, a function might declare that it
648+
requires the GIL to be held in order to use it. This refers to having an
649+
:term:`attached thread state`.
650+
625651
hash-based pyc
626652
A bytecode cache file that uses the hash rather than the last-modified
627653
time of the corresponding source file to determine its validity. See
@@ -1295,6 +1321,29 @@ Glossary
12951321
See also :term:`binary file` for a file object able to read and write
12961322
:term:`bytes-like objects <bytes-like object>`.
12971323

1324+
thread state
1325+
1326+
The information used by the :term:`CPython` runtime to run in an OS thread.
1327+
For example, this includes the current exception, if any, and the
1328+
state of the bytecode interpreter.
1329+
1330+
Each thread state is bound to a single OS thread, but threads may have
1331+
many thread states available. At most, one of them may be
1332+
:term:`attached <attached thread state>` at once.
1333+
1334+
An :term:`attached thread state` is required to call most
1335+
of Python's C API, unless a function explicitly documents otherwise.
1336+
The bytecode interpreter only runs under an attached thread state.
1337+
1338+
Each thread state belongs to a single interpreter, but each interpreter
1339+
may have many thread states, including multiple for the same OS thread.
1340+
Thread states from multiple interpreters may be bound to the same
1341+
thread, but only one can be :term:`attached <attached thread state>` in
1342+
that thread at any given moment.
1343+
1344+
See :ref:`Thread State and the Global Interpreter Lock <threads>` for more
1345+
information.
1346+
12981347
token
12991348

13001349
A small unit of source code, generated by the

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.