Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit d687900

Browse filesBrowse files
naschemekumaraditya303
nascheme
and
kumaraditya303
authored
gh-128384: Use a context variable for warnings.catch_warnings (gh-130010)
Make `warnings.catch_warnings()` use a context variable for holding the warning filtering state if the `sys.flags.context_aware_warnings` flag is set to true. This makes using the context manager thread-safe in multi-threaded programs. Add the `sys.flags.thread_inherit_context` flag. If true, starting a new thread with `threading.Thread` will use a copy of the context from the caller of `Thread.start()`. Both these flags are set to true by default for the free-threaded build and false for the default build. Move the Python implementation of warnings.py into _py_warnings.py. Make _contextvars a builtin module. Co-authored-by: Kumar Aditya <kumaraditya@python.org>
1 parent e523754 commit d687900
Copy full SHA for d687900

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Dismiss banner
Expand file treeCollapse file tree

41 files changed

+1852
-961
lines changed

‎Doc/howto/free-threading-python.rst

Copy file name to clipboardExpand all lines: Doc/howto/free-threading-python.rst
+30Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,3 +152,33 @@ to re-enable it in a thread-safe way in the 3.14 release. This overhead is
152152
expected to be reduced in upcoming Python release. We are aiming for an
153153
overhead of 10% or less on the pyperformance suite compared to the default
154154
GIL-enabled build.
155+
156+
157+
Behavioral changes
158+
==================
159+
160+
This section describes CPython behavioural changes with the free-threaded
161+
build.
162+
163+
164+
Context variables
165+
-----------------
166+
167+
In the free-threaded build, the flag :data:`~sys.flags.thread_inherit_context`
168+
is set to true by default which causes threads created with
169+
:class:`threading.Thread` to start with a copy of the
170+
:class:`~contextvars.Context()` of the caller of
171+
:meth:`~threading.Thread.start`. In the default GIL-enabled build, the flag
172+
defaults to false so threads start with an
173+
empty :class:`~contextvars.Context()`.
174+
175+
176+
Warning filters
177+
---------------
178+
179+
In the free-threaded build, the flag :data:`~sys.flags.context_aware_warnings`
180+
is set to true by default. In the default GIL-enabled build, the flag defaults
181+
to false. If the flag is true then the :class:`warnings.catch_warnings`
182+
context manager uses a context variable for warning filters. If the flag is
183+
false then :class:`~warnings.catch_warnings` modifies the global filters list,
184+
which is not thread-safe. See the :mod:`warnings` module for more details.

‎Doc/library/decimal.rst

Copy file name to clipboardExpand all lines: Doc/library/decimal.rst
+14-7Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1884,13 +1884,20 @@ the current thread.
18841884

18851885
If :func:`setcontext` has not been called before :func:`getcontext`, then
18861886
:func:`getcontext` will automatically create a new context for use in the
1887-
current thread.
1888-
1889-
The new context is copied from a prototype context called *DefaultContext*. To
1890-
control the defaults so that each thread will use the same values throughout the
1891-
application, directly modify the *DefaultContext* object. This should be done
1892-
*before* any threads are started so that there won't be a race condition between
1893-
threads calling :func:`getcontext`. For example::
1887+
current thread. New context objects have default values set from the
1888+
:data:`decimal.DefaultContext` object.
1889+
1890+
The :data:`sys.flags.thread_inherit_context` flag affects the context for
1891+
new threads. If the flag is false, new threads will start with an empty
1892+
context. In this case, :func:`getcontext` will create a new context object
1893+
when called and use the default values from *DefaultContext*. If the flag
1894+
is true, new threads will start with a copy of context from the caller of
1895+
:meth:`threading.Thread.start`.
1896+
1897+
To control the defaults so that each thread will use the same values throughout
1898+
the application, directly modify the *DefaultContext* object. This should be
1899+
done *before* any threads are started so that there won't be a race condition
1900+
between threads calling :func:`getcontext`. For example::
18941901

18951902
# Set applicationwide defaults for all threads about to be launched
18961903
DefaultContext.prec = 12

‎Doc/library/sys.rst

Copy file name to clipboardExpand all lines: Doc/library/sys.rst
+23-1Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -535,7 +535,8 @@ always available. Unless explicitly noted otherwise, all variables are read-only
535535
.. data:: flags
536536

537537
The :term:`named tuple` *flags* exposes the status of command line
538-
flags. The attributes are read only.
538+
flags. Flags should only be accessed only by name and not by index. The
539+
attributes are read only.
539540

540541
.. list-table::
541542

@@ -594,6 +595,18 @@ always available. Unless explicitly noted otherwise, all variables are read-only
594595
* - .. attribute:: flags.warn_default_encoding
595596
- :option:`-X warn_default_encoding <-X>`
596597

598+
* - .. attribute:: flags.gil
599+
- :option:`-X gil <-X>` and :envvar:`PYTHON_GIL`
600+
601+
* - .. attribute:: flags.thread_inherit_context
602+
- :option:`-X thread_inherit_context <-X>` and
603+
:envvar:`PYTHON_THREAD_INHERIT_CONTEXT`
604+
605+
* - .. attribute:: flags.context_aware_warnings
606+
- :option:`-X thread_inherit_context <-X>` and
607+
:envvar:`PYTHON_CONTEXT_AWARE_WARNINGS`
608+
609+
597610
.. versionchanged:: 3.2
598611
Added ``quiet`` attribute for the new :option:`-q` flag.
599612

@@ -620,6 +633,15 @@ always available. Unless explicitly noted otherwise, all variables are read-only
620633
.. versionchanged:: 3.11
621634
Added the ``int_max_str_digits`` attribute.
622635

636+
.. versionchanged:: 3.13
637+
Added the ``gil`` attribute.
638+
639+
.. versionchanged:: 3.14
640+
Added the ``thread_inherit_context`` attribute.
641+
642+
.. versionchanged:: 3.14
643+
Added the ``context_aware_warnings`` attribute.
644+
623645

624646
.. data:: float_info
625647

‎Doc/library/threading.rst

Copy file name to clipboardExpand all lines: Doc/library/threading.rst
+14-1Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -334,7 +334,7 @@ since it is impossible to detect the termination of alien threads.
334334

335335

336336
.. class:: Thread(group=None, target=None, name=None, args=(), kwargs={}, *, \
337-
daemon=None)
337+
daemon=None, context=None)
338338

339339
This constructor should always be called with keyword arguments. Arguments
340340
are:
@@ -359,6 +359,16 @@ since it is impossible to detect the termination of alien threads.
359359
If ``None`` (the default), the daemonic property is inherited from the
360360
current thread.
361361

362+
*context* is the :class:`~contextvars.Context` value to use when starting
363+
the thread. The default value is ``None`` which indicates that the
364+
:data:`sys.flags.thread_inherit_context` flag controls the behaviour. If
365+
the flag is true, threads will start with a copy of the context of the
366+
caller of :meth:`~Thread.start`. If false, they will start with an empty
367+
context. To explicitly start with an empty context, pass a new instance of
368+
:class:`~contextvars.Context()`. To explicitly start with a copy of the
369+
current context, pass the value from :func:`~contextvars.copy_context`. The
370+
flag defaults true on free-threaded builds and false otherwise.
371+
362372
If the subclass overrides the constructor, it must make sure to invoke the
363373
base class constructor (``Thread.__init__()``) before doing anything else to
364374
the thread.
@@ -369,6 +379,9 @@ since it is impossible to detect the termination of alien threads.
369379
.. versionchanged:: 3.10
370380
Use the *target* name if *name* argument is omitted.
371381

382+
.. versionchanged:: 3.14
383+
Added the *context* parameter.
384+
372385
.. method:: start()
373386

374387
Start the thread's activity.

‎Doc/library/warnings.rst

Copy file name to clipboardExpand all lines: Doc/library/warnings.rst
+76-12Lines changed: 76 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -324,11 +324,13 @@ the warning using the :class:`catch_warnings` context manager::
324324
While within the context manager all warnings will simply be ignored. This
325325
allows you to use known-deprecated code without having to see the warning while
326326
not suppressing the warning for other code that might not be aware of its use
327-
of deprecated code. Note: this can only be guaranteed in a single-threaded
328-
application. If two or more threads use the :class:`catch_warnings` context
329-
manager at the same time, the behavior is undefined.
327+
of deprecated code.
330328

329+
.. note::
331330

331+
See :ref:`warning-concurrent-safe` for details on the
332+
concurrency-safety of the :class:`catch_warnings` context manager when
333+
used in programs using multiple threads or async functions.
332334

333335
.. _warning-testing:
334336

@@ -364,10 +366,13 @@ the warning has been cleared.
364366
Once the context manager exits, the warnings filter is restored to its state
365367
when the context was entered. This prevents tests from changing the warnings
366368
filter in unexpected ways between tests and leading to indeterminate test
367-
results. The :func:`showwarning` function in the module is also restored to
368-
its original value. Note: this can only be guaranteed in a single-threaded
369-
application. If two or more threads use the :class:`catch_warnings` context
370-
manager at the same time, the behavior is undefined.
369+
results.
370+
371+
.. note::
372+
373+
See :ref:`warning-concurrent-safe` for details on the
374+
concurrency-safety of the :class:`catch_warnings` context manager when
375+
used in programs using multiple threads or async functions.
371376

372377
When testing multiple operations that raise the same kind of warning, it
373378
is important to test them in a manner that confirms each operation is raising
@@ -615,12 +620,71 @@ Available Context Managers
615620

616621
.. note::
617622

618-
The :class:`catch_warnings` manager works by replacing and
619-
then later restoring the module's
620-
:func:`showwarning` function and internal list of filter
621-
specifications. This means the context manager is modifying
622-
global state and therefore is not thread-safe.
623+
See :ref:`warning-concurrent-safe` for details on the
624+
concurrency-safety of the :class:`catch_warnings` context manager when
625+
used in programs using multiple threads or async functions.
626+
623627

624628
.. versionchanged:: 3.11
625629

626630
Added the *action*, *category*, *lineno*, and *append* parameters.
631+
632+
633+
.. _warning-concurrent-safe:
634+
635+
Concurrent safety of Context Managers
636+
-------------------------------------
637+
638+
The behavior of :class:`catch_warnings` context manager depends on the
639+
:data:`sys.flags.context_aware_warnings` flag. If the flag is true, the
640+
context manager behaves in a concurrent-safe fashion and otherwise not.
641+
Concurrent-safe means that it is both thread-safe and safe to use within
642+
:ref:`asyncio coroutines <coroutine>` and tasks. Being thread-safe means
643+
that behavior is predictable in a multi-threaded program. The flag defaults
644+
to true for free-threaded builds and false otherwise.
645+
646+
If the :data:`~sys.flags.context_aware_warnings` flag is false, then
647+
:class:`catch_warnings` will modify the global attributes of the
648+
:mod:`warnings` module. This is not safe if used within a concurrent program
649+
(using multiple threads or using asyncio coroutines). For example, if two
650+
or more threads use the :class:`catch_warnings` class at the same time, the
651+
behavior is undefined.
652+
653+
If the flag is true, :class:`catch_warnings` will not modify global
654+
attributes and will instead use a :class:`~contextvars.ContextVar` to
655+
store the newly established warning filtering state. A context variable
656+
provides thread-local storage and it makes the use of :class:`catch_warnings`
657+
thread-safe.
658+
659+
The *record* parameter of the context handler also behaves differently
660+
depending on the value of the flag. When *record* is true and the flag is
661+
false, the context manager works by replacing and then later restoring the
662+
module's :func:`showwarning` function. That is not concurrent-safe.
663+
664+
When *record* is true and the flag is true, the :func:`showwarning` function
665+
is not replaced. Instead, the recording status is indicated by an internal
666+
property in the context variable. In this case, the :func:`showwarning`
667+
function will not be restored when exiting the context handler.
668+
669+
The :data:`~sys.flags.context_aware_warnings` flag can be set the :option:`-X
670+
context_aware_warnings<-X>` command-line option or by the
671+
:envvar:`PYTHON_CONTEXT_AWARE_WARNINGS` environment variable.
672+
673+
.. note::
674+
675+
It is likely that most programs that desire thread-safe
676+
behaviour of the warnings module will also want to set the
677+
:data:`~sys.flags.thread_inherit_context` flag to true. That flag
678+
causes threads created by :class:`threading.Thread` to start
679+
with a copy of the context variables from the thread starting
680+
it. When true, the context established by :class:`catch_warnings`
681+
in one thread will also apply to new threads started by it. If false,
682+
new threads will start with an empty warnings context variable,
683+
meaning that any filtering that was established by a
684+
:class:`catch_warnings` context manager will no longer be active.
685+
686+
.. versionchanged:: 3.14
687+
688+
Added the :data:`sys.flags.context_aware_warnings` flag and the use of a
689+
context variable for :class:`catch_warnings` if the flag is true. Previous
690+
versions of Python acted as if the flag was always set to false.

‎Doc/using/cmdline.rst

Copy file name to clipboardExpand all lines: Doc/using/cmdline.rst
+37Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -639,6 +639,23 @@ Miscellaneous options
639639

640640
.. versionadded:: 3.13
641641

642+
* :samp:`-X thread_inherit_context={0,1}` causes :class:`~threading.Thread`
643+
to, by default, use a copy of context of of the caller of
644+
``Thread.start()`` when starting. Otherwise, threads will start
645+
with an empty context. If unset, the value of this option defaults
646+
to ``1`` on free-threaded builds and to ``0`` otherwise. See also
647+
:envvar:`PYTHON_THREAD_INHERIT_CONTEXT`.
648+
649+
.. versionadded:: 3.14
650+
651+
* :samp:`-X context_aware_warnings={0,1}` causes the
652+
:class:`warnings.catch_warnings` context manager to use a
653+
:class:`~contextvars.ContextVar` to store warnings filter state. If
654+
unset, the value of this option defaults to ``1`` on free-threaded builds
655+
and to ``0`` otherwise. See also :envvar:`PYTHON_CONTEXT_AWARE_WARNINGS`.
656+
657+
.. versionadded:: 3.14
658+
642659
It also allows passing arbitrary values and retrieving them through the
643660
:data:`sys._xoptions` dictionary.
644661

@@ -1241,6 +1258,26 @@ conflict.
12411258

12421259
.. versionadded:: 3.13
12431260

1261+
.. envvar:: PYTHON_THREAD_INHERIT_CONTEXT
1262+
1263+
If this variable is set to ``1`` then :class:`~threading.Thread` will,
1264+
by default, use a copy of context of of the caller of ``Thread.start()``
1265+
when starting. Otherwise, new threads will start with an empty context.
1266+
If unset, this variable defaults to ``1`` on free-threaded builds and to
1267+
``0`` otherwise. See also :option:`-X thread_inherit_context<-X>`.
1268+
1269+
.. versionadded:: 3.14
1270+
1271+
.. envvar:: PYTHON_CONTEXT_AWARE_WARNINGS
1272+
1273+
If set to ``1`` then the :class:`warnings.catch_warnings` context
1274+
manager will use a :class:`~contextvars.ContextVar` to store warnings
1275+
filter state. If unset, this variable defaults to ``1`` on
1276+
free-threaded builds and to ``0`` otherwise. See :option:`-X
1277+
context_aware_warnings<-X>`.
1278+
1279+
.. versionadded:: 3.14
1280+
12441281
Debug-mode variables
12451282
~~~~~~~~~~~~~~~~~~~~
12461283

‎Include/cpython/initconfig.h

Copy file name to clipboardExpand all lines: Include/cpython/initconfig.h
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -180,6 +180,8 @@ typedef struct PyConfig {
180180
int use_frozen_modules;
181181
int safe_path;
182182
int int_max_str_digits;
183+
int thread_inherit_context;
184+
int context_aware_warnings;
183185
#ifdef __APPLE__
184186
int use_system_logger;
185187
#endif

‎Include/internal/pycore_global_objects_fini_generated.h

Copy file name to clipboardExpand all lines: Include/internal/pycore_global_objects_fini_generated.h
+1Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Include/internal/pycore_global_strings.h

Copy file name to clipboardExpand all lines: Include/internal/pycore_global_strings.h
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,7 @@ struct _Py_global_strings {
244244
STRUCT_FOR_ID(_feature_version)
245245
STRUCT_FOR_ID(_field_types)
246246
STRUCT_FOR_ID(_fields_)
247+
STRUCT_FOR_ID(_filters)
247248
STRUCT_FOR_ID(_finalizing)
248249
STRUCT_FOR_ID(_find_and_load)
249250
STRUCT_FOR_ID(_fix_up_module)

‎Include/internal/pycore_interp_structs.h

Copy file name to clipboardExpand all lines: Include/internal/pycore_interp_structs.h
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -582,6 +582,7 @@ struct _warnings_runtime_state {
582582
PyObject *default_action; /* String */
583583
_PyRecursiveMutex lock;
584584
long filters_version;
585+
PyObject *context;
585586
};
586587

587588
struct _Py_mem_interp_free_queue {

‎Include/internal/pycore_runtime_init_generated.h

Copy file name to clipboardExpand all lines: Include/internal/pycore_runtime_init_generated.h
+1Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Include/internal/pycore_unicodeobject_generated.h

Copy file name to clipboardExpand all lines: Include/internal/pycore_unicodeobject_generated.h
+4Lines changed: 4 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.