From 6ecb4b33e2b533ee46d2485d3a3d2c73df1e1b5e Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 16:08:49 +1000 Subject: [PATCH 01/18] gh-74929: PEP 667 general docs update * expand on What's New entry for PEP 667 * review (and update) all mentions of "locals" in documentation C API documentation updates will be added in a separate PR --- Doc/c-api/frame.rst | 5 +- Doc/library/code.rst | 6 +-- Doc/library/functions.rst | 102 +++++++++++++++++++++++++++++--------- Doc/library/pdb.rst | 24 +++++---- Doc/library/profile.rst | 2 +- Doc/library/traceback.rst | 2 +- Doc/whatsnew/3.13.rst | 32 ++++++++++-- Lib/code.py | 2 +- 8 files changed, 129 insertions(+), 46 deletions(-) diff --git a/Doc/c-api/frame.rst b/Doc/c-api/frame.rst index 82e0980ad753d0..4d954dcd724875 100644 --- a/Doc/c-api/frame.rst +++ b/Doc/c-api/frame.rst @@ -123,8 +123,9 @@ See also :ref:`Reflection `. Get the *frame*'s :attr:`~frame.f_locals` attribute. If the frame refers to a function or comprehension, this returns a write-through proxy object that allows modifying the locals. - In all other cases (classes, modules) it returns the :class:`dict` - representing the frame locals directly. + In all other cases (classes, modules, :func:`exec`, :func:`eval`) it returns + the mapping representing the frame locals directly (as described for + :func:`locals`). Return a :term:`strong reference`. diff --git a/Doc/library/code.rst b/Doc/library/code.rst index 8c3a3e8e95a11f..46d5e4807b05ee 100644 --- a/Doc/library/code.rst +++ b/Doc/library/code.rst @@ -18,9 +18,9 @@ build applications which provide an interactive interpreter prompt. This class deals with parsing and interpreter state (the user's namespace); it does not deal with input buffering or prompting or input file naming (the filename is always passed in explicitly). The optional *locals* argument - specifies the dictionary in which code will be executed; it defaults to a newly - created dictionary with key ``'__name__'`` set to ``'__console__'`` and key - ``'__doc__'`` set to ``None``. + specifies the mapping namespace in which code will be executed; it defaults + to a newly created dictionary with key ``'__name__'`` set to + ``'__console__'`` and key ``'__doc__'`` set to ``None``. .. class:: InteractiveConsole(locals=None, filename="", local_exit=False) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 0c7ef67774cd05..49bdd89bedc00b 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -543,18 +543,19 @@ are always available. They are listed here in alphabetical order. The *expression* argument is parsed and evaluated as a Python expression (technically speaking, a condition list) using the *globals* and *locals* - dictionaries as global and local namespace. If the *globals* dictionary is + mappings as global and local namespace. If the *globals* dictionary is present and does not contain a value for the key ``__builtins__``, a reference to the dictionary of the built-in module :mod:`builtins` is inserted under that key before *expression* is parsed. That way you can control what builtins are available to the executed code by inserting your own ``__builtins__`` dictionary into *globals* before passing it to - :func:`eval`. If the *locals* dictionary is omitted it defaults to the - *globals* dictionary. If both dictionaries are omitted, the expression is + :func:`eval`. If the *locals* mapping is omitted it defaults to the + *globals* dictionary. If both mappings are omitted, the expression is executed with the *globals* and *locals* in the environment where - :func:`eval` is called. Note, *eval()* does not have access to the + :func:`eval` is called. Note, *eval()* will only have access to the :term:`nested scopes ` (non-locals) in the enclosing - environment. + environment if they are already referenced in the scope that is calling + :func:`eval` (e.g. via a :keyword:`nonlocal` statement). Example: @@ -587,6 +588,11 @@ are always available. They are listed here in alphabetical order. The *globals* and *locals* arguments can now be passed as keywords. + .. versionchanged:: 3.13 + + The semantics of the default *locals* namespace have been adjusted as + described for the :func:`locals` builtin. + .. index:: pair: built-in function; exec .. function:: exec(source, /, globals=None, locals=None, *, closure=None) @@ -608,9 +614,9 @@ are always available. They are listed here in alphabetical order. will be used for both the global and the local variables. If *globals* and *locals* are given, they are used for the global and local variables, respectively. If provided, *locals* can be any mapping object. Remember - that at the module level, globals and locals are the same dictionary. If exec - gets two separate objects as *globals* and *locals*, the code will be - executed as if it were embedded in a class definition. + that at the module level, globals and locals are the same dictionary. If + ``exec`` gets two separate objects as *globals* and *locals*, the code will + be executed as if it were embedded in a class definition. If the *globals* dictionary does not contain a value for the key ``__builtins__``, a reference to the dictionary of the built-in module @@ -631,7 +637,7 @@ are always available. They are listed here in alphabetical order. .. note:: The built-in functions :func:`globals` and :func:`locals` return the current - global and local dictionary, respectively, which may be useful to pass around + global and local namespace, respectively, which may be useful to pass around for use as the second and third argument to :func:`exec`. .. note:: @@ -647,6 +653,11 @@ are always available. They are listed here in alphabetical order. The *globals* and *locals* arguments can now be passed as keywords. + .. versionchanged:: 3.13 + + The semantics of the default *locals* namespace have been adjusted as + described for the :func:`locals` builtin. + .. function:: filter(function, iterable) @@ -1063,10 +1074,10 @@ are always available. They are listed here in alphabetical order. In all of the above cases, each call to ``locals()`` in a given frame of execution will return the *same* mapping object. Changes made through - the mapping object returned from ``locals()`` will be visible as bound, - rebound, or deleted local variables, and binding, rebinding, or deleting - local variables will immediately affect the contents of the returned mapping - object. + the mapping object returned from ``locals()`` will be visible as assigned, + reassigned, or deleted local variables, and assigning, reassigning, or + deleting local variables will immediately affect the contents of the + returned mapping object. At function scope (including for generators and coroutines), each call to ``locals()`` instead returns a fresh dictionary containing the current @@ -1077,15 +1088,56 @@ are always available. They are listed here in alphabetical order. cell references does *not* affect the contents of previously returned dictionaries. + Calling ``locals()`` as part of a comprehension in a function, generator, or + coroutine is equivalent to calling it in the containing scope, except that + the comprehension's initialised iteration variables will be included. In + other scopes, it behaves as if the comprehension were running as a nested + function. + + Calling ``locals()`` as part of a generator expression is equivalent to + calling it in a nested generator function. + + .. versionchanged:: 3.13 - In previous versions, the semantics of mutating the mapping object - returned from this function were formally undefined. In CPython - specifically, the mapping returned at function scope could be - implicitly refreshed by other operations, such as calling ``locals()`` - again. Obtaining the legacy CPython behaviour now requires explicit - calls to update the initially returned dictionary with the results - of subsequent calls to ``locals()``. + As part of :pep:`667`, the semantics of mutating the mapping object + returned from this function are now formally defined. The behavior in + functions, generators, coroutines, comprehensions, and generator + expressions is now as described above. Aside from being defined, the + behaviour in other scopes remains unchanged from previous versions. + .. versionchanged:: 3.13 + As part of :pep:`667`, the mapping returned when called in a function, + generator, coroutine, comprehension or generator expression is an + independent snapshot of the currently assigned local and locally + referenced nonlocal variables. In CPython specifically, the mapping + returned in these scopes could previously be implicitly refreshed by other + operations, such as calling ``locals()`` again. Obtaining the legacy + CPython behaviour now requires explicit calls to update the initially + returned dictionary with the results of subsequent calls to ``locals()``. + + .. versionchanged:: 3.13 + As part of :pep:`667`, calling `locals()` in a comprehension at module + or class scope (including via `exec` or `eval`) once more behaves as if + the comprehension were running as an independent nested function (i.e. + the local variables from the containing scope are not included). + + .. versionchanged:: 3.12 + As part of :pep:`709`, calling `locals()` in a comprehension at module + or class scope (including via `exec` or `eval`) produces an independent + snapshot of the containing scope with the comprehension iteration + variable(s) included. + + .. versionchanged:: 3.12 + As part of :pep:`709`, calling `locals()` in a comprehension as part of a + function, generator, or coroutine returns the same `locals()` reference + as calls in the containing scope. + + .. + Possible simplification: change the 3.12 note to just "Changes to + behavior in comprehensions as described in :pep:`709`", and do somethine + similar for 3.13 and :pep:`667`. While the draft change notes are + comprehensive, they're also almost entirely in the "Who is really going + to care?" category. .. function:: map(function, iterable, *iterables) @@ -1971,14 +2023,18 @@ are always available. They are listed here in alphabetical order. :attr:`~object.__dict__` attributes (for example, classes use a :class:`types.MappingProxyType` to prevent direct dictionary updates). - Without an argument, :func:`vars` acts like :func:`locals`. Note, the - locals dictionary is only useful for reads since updates to the locals - dictionary are ignored. + Without an argument, :func:`vars` acts like :func:`locals`. A :exc:`TypeError` exception is raised if an object is specified but it doesn't have a :attr:`~object.__dict__` attribute (for example, if its class defines the :attr:`~object.__slots__` attribute). + .. versionchanged:: 3.13 + + The semantics of calling this function without an argument have been + adjusted as described for the :func:`locals` builtin. + + .. function:: zip(*iterables, strict=False) Iterate over several iterables in parallel, producing tuples with an item diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst index 7a47a7d5d6754f..739c466fe2c21c 100644 --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -123,6 +123,11 @@ The typical usage to inspect a crashed program is:: 0 (Pdb) +.. versionchanged:: 3.13 + The implementation of :pep:`667` means that name assignments made via `pdb` + will consistently and directly affect the active scope, even when running + inside a function, generator, or coroutine. + The module defines the following functions; each enters the debugger in a slightly different way: @@ -580,17 +585,9 @@ can be overridden by the local file. .. pdbcommand:: interact Start an interactive interpreter (using the :mod:`code` module) whose global - namespace contains all the (global and local) names found in the current - scope. Use ``exit()`` or ``quit()`` to exit the interpreter and return to - the debugger. - - .. note:: - - Because interact creates a new global namespace with the current global - and local namespace for execution, assignment to variables will not - affect the original namespaces. - However, modification to the mutable objects will be reflected in the - original namespaces. + and local namespaces correspond to the global and local namespaces in the + current scope. Use ``exit()`` or ``quit()`` to exit the interpreter and + return to the debugger. .. versionadded:: 3.2 @@ -602,6 +599,11 @@ can be overridden by the local file. :pdbcmd:`interact` directs its output to the debugger's output channel rather than :data:`sys.stderr`. + .. versionchanged:: 3.13 + The implementation of :pep:`667` means that name assignments made via the + interactive console will directly affect the active scope, even when + running inside a function, generator, or coroutine. + .. _debugger-aliases: .. pdbcommand:: alias [name [command]] diff --git a/Doc/library/profile.rst b/Doc/library/profile.rst index 3ca802e024bc27..9721da7220d54d 100644 --- a/Doc/library/profile.rst +++ b/Doc/library/profile.rst @@ -234,7 +234,7 @@ functions: .. function:: runctx(command, globals, locals, filename=None, sort=-1) This function is similar to :func:`run`, with added arguments to supply the - globals and locals dictionaries for the *command* string. This routine + globals and locals mappings for the *command* string. This routine executes:: exec(command, globals, locals) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst index 9983b8da427a2a..bfd2c3efc4b1f6 100644 --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -473,7 +473,7 @@ in a :ref:`traceback `. attribute accessed (which also happens when casting it to a :class:`tuple`). :attr:`~FrameSummary.line` may be directly provided, and will prevent line lookups happening at all. *locals* is an optional local variable - dictionary, and if supplied the variable representations are stored in the + mapping, and if supplied the variable representations are stored in the summary for later display. :class:`!FrameSummary` instances have the following attributes: diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index effa554bfe8469..ae7a62fb033eb5 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -94,10 +94,10 @@ Interpreter improvements: Performance improvements are modest -- we expect to be improving this over the next few releases. -* :pep:`667`: :attr:`FrameType.f_locals ` when used in - a function now returns a write-through proxy to the frame's locals, - rather than a ``dict``. See the PEP for corresponding C API changes - and deprecations. +* :pep:`667`: The `func:`locals` builtin now has defined semantics when mutating + the returned mapping. Python debuggers and similar tools may now more reliably + update local variables in optimised frames even during concurrent code + execution. New typing features: @@ -370,6 +370,30 @@ disabled with the :envvar:`PYTHON_GIL` environment variable or the pip 24.1b1 or newer is required to install packages with C extensions in the free-threaded build. + +Defined mutation semantics for ``locals()`` +------------------------------------------- + +Historically, the semantics for mutating the return value of :func:`locals` have +been left to individual Python implementations to define. + +Through :pep:`667`, Python 3.13 standardises the historical behaviour of CPython +for most code execution scopes, but changes optimised scopes (functions, +generators, coroutines, comprehensions, generator expressions) to explicitly +return independent snapshots of the currently assigned local and nonlocal +name references. + +To ensure debuggers and similar tools can reliably update local variables in +affected scopes, :attr:`FrameType.f_locals ` now returns a +write-through proxy to the frame's local and locally reference nonlocal +variables rather than a ``dict`` with unclear runtime semantics. + +See :pep:`667` for more details, including related C API changes and +deprecations. + +.. todo: Add notes to porting guide + + Other Language Changes ====================== diff --git a/Lib/code.py b/Lib/code.py index 9d124563f728c2..2c1c12d644542f 100644 --- a/Lib/code.py +++ b/Lib/code.py @@ -25,7 +25,7 @@ class InteractiveInterpreter: def __init__(self, locals=None): """Constructor. - The optional 'locals' argument specifies the dictionary in + The optional 'locals' argument specifies the mapping in which code will be executed; it defaults to a newly created dictionary with key "__name__" set to "__console__" and key "__doc__" set to None. From fa4172b84bd7bf7bc36e39cd26161cc66764172d Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 19:13:27 +1000 Subject: [PATCH 02/18] pdb's interact still uses a new namespace --- Doc/library/pdb.rst | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst index 739c466fe2c21c..45240fd0ae26f8 100644 --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -584,11 +584,18 @@ can be overridden by the local file. .. pdbcommand:: interact - Start an interactive interpreter (using the :mod:`code` module) whose global - and local namespaces correspond to the global and local namespaces in the + Start an interactive interpreter (using the :mod:`code` module) in a new + global namespace initialised from the local and global namespaces for the current scope. Use ``exit()`` or ``quit()`` to exit the interpreter and return to the debugger. + .. note:: + + As ``interact`` creates a new dedicated namespace for code execution, + assignments to variables will not affect the original namespaces. + However, modifications to any referenced mutable objects will be reflected + in the original namespaces as usual. + .. versionadded:: 3.2 .. versionchanged:: 3.13 @@ -599,11 +606,6 @@ can be overridden by the local file. :pdbcmd:`interact` directs its output to the debugger's output channel rather than :data:`sys.stderr`. - .. versionchanged:: 3.13 - The implementation of :pep:`667` means that name assignments made via the - interactive console will directly affect the active scope, even when - running inside a function, generator, or coroutine. - .. _debugger-aliases: .. pdbcommand:: alias [name [command]] From 9ba869be9087520fdfc0ff615e8955bb9cdab974 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 19:17:34 +1000 Subject: [PATCH 03/18] Cleanup Sphinx lint errors --- Doc/library/functions.rst | 19 ++++++++++--------- Doc/library/pdb.rst | 2 +- Doc/whatsnew/3.13.rst | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 49bdd89bedc00b..d2b6b634385cab 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1063,14 +1063,15 @@ are always available. They are listed here in alphabetical order. variable names as the keys, and their currently bound references as the values. - At module scope, as well as when using ``exec()`` or ``eval()`` with a - single namespace, this function returns the same namespace as ``globals()``. + At module scope, as well as when using :func:`exec` or :func:`eval` with + a single namespace, this function returns the same namespace as + :func:`globals`. At class scope, it returns the namespace that will be passed to the metaclass constructor. When using ``exec()`` or ``eval()`` with separate local and global - namespaces, it returns the local namespace passed in to the function call. + arguments, it returns the local namespace passed in to the function call. In all of the above cases, each call to ``locals()`` in a given frame of execution will return the *same* mapping object. Changes made through @@ -1116,20 +1117,20 @@ are always available. They are listed here in alphabetical order. returned dictionary with the results of subsequent calls to ``locals()``. .. versionchanged:: 3.13 - As part of :pep:`667`, calling `locals()` in a comprehension at module - or class scope (including via `exec` or `eval`) once more behaves as if + As part of :pep:`667`, calling ``locals()`` in a comprehension at module + or class scope (including via ``exec`` or ``eval``) once more behaves as if the comprehension were running as an independent nested function (i.e. the local variables from the containing scope are not included). .. versionchanged:: 3.12 - As part of :pep:`709`, calling `locals()` in a comprehension at module - or class scope (including via `exec` or `eval`) produces an independent + As part of :pep:`709`, calling ``locals()`` in a comprehension at module + or class scope (including via ``exec`` or ``eval``) produces an independent snapshot of the containing scope with the comprehension iteration variable(s) included. .. versionchanged:: 3.12 - As part of :pep:`709`, calling `locals()` in a comprehension as part of a - function, generator, or coroutine returns the same `locals()` reference + As part of :pep:`709`, calling ``locals()`` in a comprehension as part of a + function, generator, or coroutine returns the same ``locals()`` reference as calls in the containing scope. .. diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst index 45240fd0ae26f8..293c941431c03f 100644 --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -124,7 +124,7 @@ The typical usage to inspect a crashed program is:: (Pdb) .. versionchanged:: 3.13 - The implementation of :pep:`667` means that name assignments made via `pdb` + The implementation of :pep:`667` means that name assignments made via ``pdb`` will consistently and directly affect the active scope, even when running inside a function, generator, or coroutine. diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index ae7a62fb033eb5..272022ee4b0b09 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -94,7 +94,7 @@ Interpreter improvements: Performance improvements are modest -- we expect to be improving this over the next few releases. -* :pep:`667`: The `func:`locals` builtin now has defined semantics when mutating +* :pep:`667`: The :func:`locals` builtin now has defined semantics when mutating the returned mapping. Python debuggers and similar tools may now more reliably update local variables in optimised frames even during concurrent code execution. From 40d0a813143a4416793a6b32c869ee99000618bf Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 19:23:27 +1000 Subject: [PATCH 04/18] Improve What's New wording --- Doc/whatsnew/3.13.rst | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 272022ee4b0b09..bc73b4d67a7891 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -379,14 +379,15 @@ been left to individual Python implementations to define. Through :pep:`667`, Python 3.13 standardises the historical behaviour of CPython for most code execution scopes, but changes optimised scopes (functions, -generators, coroutines, comprehensions, generator expressions) to explicitly -return independent snapshots of the currently assigned local and nonlocal -name references. +generators, coroutines, comprehensions, and generator expressions) to explicitly +return independent snapshots of the currently assigned local variables, included +locally referenced nonlocal variables captured in closures. To ensure debuggers and similar tools can reliably update local variables in -affected scopes, :attr:`FrameType.f_locals ` now returns a -write-through proxy to the frame's local and locally reference nonlocal -variables rather than a ``dict`` with unclear runtime semantics. +scopes affected by this change, :attr:`FrameType.f_locals ` now +returns a write-through proxy to the frame's local and locally referenced +nonlocal variables in these scopes, rather than returning an inconsistently +updated shared ``dict`` instance with undefined runtime semantics. See :pep:`667` for more details, including related C API changes and deprecations. From b95890daeb18763c7355b3ee85685520809388a2 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 19:53:45 +1000 Subject: [PATCH 05/18] Define 'optimised scope' term, review f_locals --- Doc/c-api/frame.rst | 6 +++--- Doc/glossary.rst | 7 +++++++ Doc/library/functions.rst | 17 ++++++++--------- Doc/reference/datamodel.rst | 2 +- Doc/whatsnew/3.13.rst | 11 +++++------ Lib/pdb.py | 9 ++++++--- 6 files changed, 30 insertions(+), 22 deletions(-) diff --git a/Doc/c-api/frame.rst b/Doc/c-api/frame.rst index 4d954dcd724875..66499d3a012890 100644 --- a/Doc/c-api/frame.rst +++ b/Doc/c-api/frame.rst @@ -121,8 +121,8 @@ See also :ref:`Reflection `. .. c:function:: PyObject* PyFrame_GetLocals(PyFrameObject *frame) Get the *frame*'s :attr:`~frame.f_locals` attribute. - If the frame refers to a function or comprehension, this returns - a write-through proxy object that allows modifying the locals. + If the frame refers to an :term:`optimised scope`, this returns a + write-through proxy object that allows modifying the locals. In all other cases (classes, modules, :func:`exec`, :func:`eval`) it returns the mapping representing the frame locals directly (as described for :func:`locals`). @@ -132,7 +132,7 @@ See also :ref:`Reflection `. .. versionadded:: 3.11 .. versionchanged:: 3.13 - Return a proxy object for functions and comprehensions. + As part of :pep:`667`, return a proxy object for optimised scopes. .. c:function:: int PyFrame_GetLineNumber(PyFrameObject *frame) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 2846f77feb112d..3c8969374af984 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -889,6 +889,13 @@ Glossary (methods). Also the ultimate base class of any :term:`new-style class`. + optimised scope + A scope where target local variable names are reliably known to the + compiler when the code is compiled, allowing optimisation of read and + write access to these names. The local namespaces for functions, + generators, coroutines, comprehensions, and generator expressions are + optimised. + package A Python :term:`module` which can contain submodules or recursively, subpackages. Technically, a package is a Python module with a diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index d2b6b634385cab..74df31c3acbfd5 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1080,14 +1080,14 @@ are always available. They are listed here in alphabetical order. deleting local variables will immediately affect the contents of the returned mapping object. - At function scope (including for generators and coroutines), each call to - ``locals()`` instead returns a fresh dictionary containing the current - bindings of the function's local variables and any nonlocal cell references. - In this case, name binding changes made via the returned dict are *not* - written back to the corresponding local variables or nonlocal cell - references, and binding, rebinding, or deleting local variables and nonlocal - cell references does *not* affect the contents of previously returned - dictionaries. + In an :term:`optimised scope` (including functions, generators, and + coroutines), each call to ``locals()`` instead returns a fresh dictionary + containing the current bindings of the function's local variables and any + nonlocal cell references. In this case, name binding changes made via the + returned dict are *not* written back to the corresponding local variables + or nonlocal cell references, and assigning, reassigning, or deleting local + variables and nonlocal cell references does *not* affect the contents + of previously returned dictionaries. Calling ``locals()`` as part of a comprehension in a function, generator, or coroutine is equivalent to calling it in the containing scope, except that @@ -1098,7 +1098,6 @@ are always available. They are listed here in alphabetical order. Calling ``locals()`` as part of a generator expression is equivalent to calling it in a nested generator function. - .. versionchanged:: 3.13 As part of :pep:`667`, the semantics of mutating the mapping object returned from this function are now formally defined. The behavior in diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index d3e066797f8837..02680fc63ebe32 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1349,7 +1349,7 @@ Special read-only attributes * - .. attribute:: frame.f_locals - The dictionary used by the frame to look up :ref:`local variables `. - If the frame refers to a function or comprehension, + If the frame refers to an :term:`optimised scope`, this may return a write-through proxy object. .. versionchanged:: 3.13 diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index bc73b4d67a7891..d16910de599e6c 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -378,10 +378,11 @@ Historically, the semantics for mutating the return value of :func:`locals` have been left to individual Python implementations to define. Through :pep:`667`, Python 3.13 standardises the historical behaviour of CPython -for most code execution scopes, but changes optimised scopes (functions, -generators, coroutines, comprehensions, and generator expressions) to explicitly -return independent snapshots of the currently assigned local variables, included -locally referenced nonlocal variables captured in closures. +for most code execution scopes, but changes +:term:`optimised scopes ` (functions, generators, coroutines, +comprehensions, and generator expressions) to explicitly return independent +snapshots of the currently assigned local variables, including locally +referenced nonlocal variables captured in closures. To ensure debuggers and similar tools can reliably update local variables in scopes affected by this change, :attr:`FrameType.f_locals ` now @@ -392,8 +393,6 @@ updated shared ``dict`` instance with undefined runtime semantics. See :pep:`667` for more details, including related C API changes and deprecations. -.. todo: Add notes to porting guide - Other Language Changes ====================== diff --git a/Lib/pdb.py b/Lib/pdb.py index e507a9bb896611..b4e3f1f4d58596 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -392,9 +392,12 @@ def setup(self, f, tb): self.tb_lineno[tb.tb_frame] = lineno tb = tb.tb_next self.curframe = self.stack[self.curindex][0] - # The f_locals dictionary is updated from the actual frame - # locals whenever the .f_locals accessor is called, so we - # cache it here to ensure that modifications are not overwritten. + # The f_locals dictionary used to be updated from the actual frame + # locals whenever the .f_locals accessor was called, so it was + # cached here to ensure that modifications were not overwritten. While + # the caching is no longer required now that f_locals is a direct proxy + # on optimised frames, it's also harmless, so the code structure has + # been left unchanged. self.curframe_locals = self.curframe.f_locals self.set_convenience_variable(self.curframe, '_frame', self.curframe) From a87f1d73a28198845f7512269b3388ffa988d669 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 20:14:09 +1000 Subject: [PATCH 06/18] Move What's New entry to correct section, add credits --- Doc/whatsnew/3.13.rst | 50 +++++++++++++++++++++++-------------------- 1 file changed, 27 insertions(+), 23 deletions(-) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index d16910de599e6c..14efab02dfe7ab 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -247,6 +247,33 @@ Improved Error Messages through ``self.X`` from any function in its body. (Contributed by Irit Katriel in :gh:`115775`.) + +Defined mutation semantics for ``locals()`` +------------------------------------------- + +Historically, the semantics for mutating the return value of :func:`locals` have +been left to individual Python implementations to define. + +Through :pep:`667`, Python 3.13 standardises the historical behaviour of CPython +for most code execution scopes, but changes +:term:`optimised scopes ` (functions, generators, coroutines, +comprehensions, and generator expressions) to explicitly return independent +snapshots of the currently assigned local variables, including locally +referenced nonlocal variables captured in closures. + +To ensure debuggers and similar tools can reliably update local variables in +scopes affected by this change, :attr:`FrameType.f_locals ` now +returns a write-through proxy to the frame's local and locally referenced +nonlocal variables in these scopes, rather than returning an inconsistently +updated shared ``dict`` instance with undefined runtime semantics. + +See :pep:`667` for more details, including related C API changes and +deprecations. + +(PEP and implementation contributed by Mark Shannon and Tian Gao in +:gh:`74929`. Documentation updates provided by Guido van Rossum and +Alyssa Coghlan.) + Incremental Garbage Collection ------------------------------ @@ -371,29 +398,6 @@ pip 24.1b1 or newer is required to install packages with C extensions in the free-threaded build. -Defined mutation semantics for ``locals()`` -------------------------------------------- - -Historically, the semantics for mutating the return value of :func:`locals` have -been left to individual Python implementations to define. - -Through :pep:`667`, Python 3.13 standardises the historical behaviour of CPython -for most code execution scopes, but changes -:term:`optimised scopes ` (functions, generators, coroutines, -comprehensions, and generator expressions) to explicitly return independent -snapshots of the currently assigned local variables, including locally -referenced nonlocal variables captured in closures. - -To ensure debuggers and similar tools can reliably update local variables in -scopes affected by this change, :attr:`FrameType.f_locals ` now -returns a write-through proxy to the frame's local and locally referenced -nonlocal variables in these scopes, rather than returning an inconsistently -updated shared ``dict`` instance with undefined runtime semantics. - -See :pep:`667` for more details, including related C API changes and -deprecations. - - Other Language Changes ====================== From 6e3f1d1149a3dadda8752198abdfe8a6f871f8df Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 20:15:30 +1000 Subject: [PATCH 07/18] Add porting notes --- Doc/whatsnew/3.13.rst | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 14efab02dfe7ab..85a7a17d413b23 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -2215,6 +2215,24 @@ Changes in the Python API Calling it with argument ``2`` restores the current default behavior. (Contributed by Serhiy Storchaka in :gh:`66410`.) +* Calling :func:`locals` in an :term:`optimised scope` now produces an + independent snapshot on each call, and hence no longer implicitly updates + previously returned references. Obtaining the legacy CPython behaviour now + requires explicit calls to update the initially returned dictionary with the + results of subsequent calls to ``locals()``. (Changed as part of :pep:`667`.) + +* Calling :func:`locals` from a comprehension at module or class scope + (including via ``exec`` or ``eval``) once more behaves as if the comprehension + were running as an independent nested function (i.e. the local variables from + the containing scope are not included). In Python 3.12, this had changed + to include the local variables from the containing scope when implementing + :pep:`709`. (Changed as part of :pep:`667`.) + +* Accessing :attr:`FrameType.f_locals ` in an + :term:`optimised scope` now returns a write-through proxy rather than a + snapshot that gets updated at ill-specified times. If a snapshot is desired, + it must be created explicitly with ``dict`` or the proxy's ``.copy()`` method. + (Changed as part of :pep:`667`.) Changes in the C API -------------------- From ebcf2d0e2f72c80e8e61ca70f9518ee7b00277a1 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 20:24:15 +1000 Subject: [PATCH 08/18] Simplify locals() versionchanged notes --- Doc/library/functions.rst | 45 +++++++-------------------------------- 1 file changed, 8 insertions(+), 37 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 74df31c3acbfd5..ecc2d677ca1f14 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1098,46 +1098,17 @@ are always available. They are listed here in alphabetical order. Calling ``locals()`` as part of a generator expression is equivalent to calling it in a nested generator function. - .. versionchanged:: 3.13 - As part of :pep:`667`, the semantics of mutating the mapping object - returned from this function are now formally defined. The behavior in - functions, generators, coroutines, comprehensions, and generator - expressions is now as described above. Aside from being defined, the - behaviour in other scopes remains unchanged from previous versions. - - .. versionchanged:: 3.13 - As part of :pep:`667`, the mapping returned when called in a function, - generator, coroutine, comprehension or generator expression is an - independent snapshot of the currently assigned local and locally - referenced nonlocal variables. In CPython specifically, the mapping - returned in these scopes could previously be implicitly refreshed by other - operations, such as calling ``locals()`` again. Obtaining the legacy - CPython behaviour now requires explicit calls to update the initially - returned dictionary with the results of subsequent calls to ``locals()``. + .. versionchanged:: 3.12 + The behaviour of ``locals()`` in a comprehension has been updated as + described in :pep:`709`. .. versionchanged:: 3.13 - As part of :pep:`667`, calling ``locals()`` in a comprehension at module - or class scope (including via ``exec`` or ``eval``) once more behaves as if - the comprehension were running as an independent nested function (i.e. - the local variables from the containing scope are not included). + As part of :pep:`667`, the semantics of mutating the mapping objects + returned from this function are now defined. The behavior in + :term:`optimised scopes ` is now as described above. + Aside from being defined, the behaviour in other scopes remains + unchanged from previous versions. - .. versionchanged:: 3.12 - As part of :pep:`709`, calling ``locals()`` in a comprehension at module - or class scope (including via ``exec`` or ``eval``) produces an independent - snapshot of the containing scope with the comprehension iteration - variable(s) included. - - .. versionchanged:: 3.12 - As part of :pep:`709`, calling ``locals()`` in a comprehension as part of a - function, generator, or coroutine returns the same ``locals()`` reference - as calls in the containing scope. - - .. - Possible simplification: change the 3.12 note to just "Changes to - behavior in comprehensions as described in :pep:`709`", and do somethine - similar for 3.13 and :pep:`667`. While the draft change notes are - comprehensive, they're also almost entirely in the "Who is really going - to care?" category. .. function:: map(function, iterable, *iterables) From 1e734608a7d8b9286419342667bdc3009d3ee3c2 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 20:30:56 +1000 Subject: [PATCH 09/18] Use newly defined term in pdb docs --- Doc/library/pdb.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst index 293c941431c03f..6e38ddd35fa332 100644 --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -126,7 +126,7 @@ The typical usage to inspect a crashed program is:: .. versionchanged:: 3.13 The implementation of :pep:`667` means that name assignments made via ``pdb`` will consistently and directly affect the active scope, even when running - inside a function, generator, or coroutine. + inside an :term:`optimised scope`. The module defines the following functions; each enters the debugger in a From a867005f0eeb46fe4a1398d33ac2eaef0c68cb7c Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Mon, 20 May 2024 20:31:14 +1000 Subject: [PATCH 10/18] Revert extraneous whitespace change --- Doc/whatsnew/3.13.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 85a7a17d413b23..e88702a605be90 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -397,7 +397,6 @@ disabled with the :envvar:`PYTHON_GIL` environment variable or the pip 24.1b1 or newer is required to install packages with C extensions in the free-threaded build. - Other Language Changes ====================== From c5cb0015691deed37bbd53679ba36a170a775e26 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 21 May 2024 11:59:18 +1000 Subject: [PATCH 11/18] Replace the 'mapping namespace' phrase --- Lib/code.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Lib/code.py b/Lib/code.py index 2c1c12d644542f..0c2fd2963b2118 100644 --- a/Lib/code.py +++ b/Lib/code.py @@ -25,10 +25,10 @@ class InteractiveInterpreter: def __init__(self, locals=None): """Constructor. - The optional 'locals' argument specifies the mapping in - which code will be executed; it defaults to a newly created - dictionary with key "__name__" set to "__console__" and key - "__doc__" set to None. + The optional 'locals' argument specifies a mapping to use as the + namespace in which code will be executed; it defaults to a newly + created dictionary with key "__name__" set to "__console__" and + key "__doc__" set to None. """ if locals is None: From d0fa5e13b19a5b41ef89e6cf97c01f316bc1b1fb Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 21 May 2024 12:21:27 +1000 Subject: [PATCH 12/18] Review comments --- Doc/library/functions.rst | 4 ++-- Doc/whatsnew/3.13.rst | 12 +++++++----- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 2e3f9cb45f8168..22609d9ae6edbd 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -2012,8 +2012,8 @@ are always available. They are listed here in alphabetical order. .. versionchanged:: 3.13 - The semantics of calling this function without an argument have been - adjusted as described for the :func:`locals` builtin. + The result of calling this function without an argument has been + updated as described for the :func:`locals` builtin. .. function:: zip(*iterables, strict=False) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 074660e15a7154..ca586511cc9bb5 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -94,9 +94,10 @@ Interpreter improvements: Performance improvements are modest -- we expect to be improving this over the next few releases. -* :pep:`667`: The :func:`locals` builtin now has defined semantics when mutating - the returned mapping. Python debuggers and similar tools may now more reliably - update local variables in optimised frames even during concurrent code +* :pep:`667`: The :func:`locals` builtin now has + :ref:`defined semantics ` when mutating the + returned mapping. Python debuggers and similar tools may now more reliably + update local variables in optimized frames even during concurrent code execution. New typing features: @@ -247,12 +248,13 @@ Improved Error Messages through ``self.X`` from any function in its body. (Contributed by Irit Katriel in :gh:`115775`.) +.. _whatsnew313-locals-semantics: Defined mutation semantics for ``locals()`` ------------------------------------------- -Historically, the semantics for mutating the return value of :func:`locals` have -been left to individual Python implementations to define. +Historically, the expected result of mutating the return value of :func:`locals` +has been left to individual Python implementations to define. Through :pep:`667`, Python 3.13 standardises the historical behaviour of CPython for most code execution scopes, but changes From e109eeeaf47486be1b3b372e7fc78da5cd785490 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 21 May 2024 12:22:14 +1000 Subject: [PATCH 13/18] Use US spelling for 'optimized' --- Doc/c-api/frame.rst | 4 ++-- Doc/glossary.rst | 6 +++--- Doc/library/functions.rst | 4 ++-- Doc/library/pdb.rst | 4 ++-- Doc/reference/datamodel.rst | 2 +- Doc/whatsnew/3.13.rst | 6 +++--- Lib/pdb.py | 2 +- Lib/test/test_gdb/util.py | 2 +- Misc/HISTORY | 2 +- Misc/NEWS.d/3.11.0a1.rst | 2 +- 10 files changed, 17 insertions(+), 17 deletions(-) diff --git a/Doc/c-api/frame.rst b/Doc/c-api/frame.rst index 66499d3a012890..638a740e0c24da 100644 --- a/Doc/c-api/frame.rst +++ b/Doc/c-api/frame.rst @@ -121,7 +121,7 @@ See also :ref:`Reflection `. .. c:function:: PyObject* PyFrame_GetLocals(PyFrameObject *frame) Get the *frame*'s :attr:`~frame.f_locals` attribute. - If the frame refers to an :term:`optimised scope`, this returns a + If the frame refers to an :term:`optimized scope`, this returns a write-through proxy object that allows modifying the locals. In all other cases (classes, modules, :func:`exec`, :func:`eval`) it returns the mapping representing the frame locals directly (as described for @@ -132,7 +132,7 @@ See also :ref:`Reflection `. .. versionadded:: 3.11 .. versionchanged:: 3.13 - As part of :pep:`667`, return a proxy object for optimised scopes. + As part of :pep:`667`, return a proxy object for optimized scopes. .. c:function:: int PyFrame_GetLineNumber(PyFrameObject *frame) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 3c8969374af984..a586145a0c5554 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -889,12 +889,12 @@ Glossary (methods). Also the ultimate base class of any :term:`new-style class`. - optimised scope + optimized scope A scope where target local variable names are reliably known to the - compiler when the code is compiled, allowing optimisation of read and + compiler when the code is compiled, allowing optimization of read and write access to these names. The local namespaces for functions, generators, coroutines, comprehensions, and generator expressions are - optimised. + optimized in this fashion. package A Python :term:`module` which can contain submodules or recursively, diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 22609d9ae6edbd..c50ef21640a5c0 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1090,7 +1090,7 @@ are always available. They are listed here in alphabetical order. deleting local variables will immediately affect the contents of the returned mapping object. - In an :term:`optimised scope` (including functions, generators, and + In an :term:`optimized scope` (including functions, generators, and coroutines), each call to ``locals()`` instead returns a fresh dictionary containing the current bindings of the function's local variables and any nonlocal cell references. In this case, name binding changes made via the @@ -1115,7 +1115,7 @@ are always available. They are listed here in alphabetical order. .. versionchanged:: 3.13 As part of :pep:`667`, the semantics of mutating the mapping objects returned from this function are now defined. The behavior in - :term:`optimised scopes ` is now as described above. + :term:`optimized scopes ` is now as described above. Aside from being defined, the behaviour in other scopes remains unchanged from previous versions. diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst index 6e38ddd35fa332..7d67e06434b799 100644 --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -125,8 +125,8 @@ The typical usage to inspect a crashed program is:: .. versionchanged:: 3.13 The implementation of :pep:`667` means that name assignments made via ``pdb`` - will consistently and directly affect the active scope, even when running - inside an :term:`optimised scope`. + will immediately affect the active scope, even when running inside an + :term:`optimized scope`. The module defines the following functions; each enters the debugger in a diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 02680fc63ebe32..0fe9681f93f135 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1349,7 +1349,7 @@ Special read-only attributes * - .. attribute:: frame.f_locals - The dictionary used by the frame to look up :ref:`local variables `. - If the frame refers to an :term:`optimised scope`, + If the frame refers to an :term:`optimized scope`, this may return a write-through proxy object. .. versionchanged:: 3.13 diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index ca586511cc9bb5..8cd174f01ff600 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -258,7 +258,7 @@ has been left to individual Python implementations to define. Through :pep:`667`, Python 3.13 standardises the historical behaviour of CPython for most code execution scopes, but changes -:term:`optimised scopes ` (functions, generators, coroutines, +:term:`optimized scopes ` (functions, generators, coroutines, comprehensions, and generator expressions) to explicitly return independent snapshots of the currently assigned local variables, including locally referenced nonlocal variables captured in closures. @@ -2206,7 +2206,7 @@ Changes in the Python API returned by :meth:`zipfile.ZipFile.open` was changed from ``'r'`` to ``'rb'``. (Contributed by Serhiy Storchaka in :gh:`115961`.) -* Calling :func:`locals` in an :term:`optimised scope` now produces an +* Calling :func:`locals` in an :term:`optimized scope` now produces an independent snapshot on each call, and hence no longer implicitly updates previously returned references. Obtaining the legacy CPython behaviour now requires explicit calls to update the initially returned dictionary with the @@ -2220,7 +2220,7 @@ Changes in the Python API :pep:`709`. (Changed as part of :pep:`667`.) * Accessing :attr:`FrameType.f_locals ` in an - :term:`optimised scope` now returns a write-through proxy rather than a + :term:`optimized scope` now returns a write-through proxy rather than a snapshot that gets updated at ill-specified times. If a snapshot is desired, it must be created explicitly with ``dict`` or the proxy's ``.copy()`` method. (Changed as part of :pep:`667`.) diff --git a/Lib/pdb.py b/Lib/pdb.py index b4e3f1f4d58596..0f791d71950099 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -396,7 +396,7 @@ def setup(self, f, tb): # locals whenever the .f_locals accessor was called, so it was # cached here to ensure that modifications were not overwritten. While # the caching is no longer required now that f_locals is a direct proxy - # on optimised frames, it's also harmless, so the code structure has + # on optimized frames, it's also harmless, so the code structure has # been left unchanged. self.curframe_locals = self.curframe.f_locals self.set_convenience_variable(self.curframe, '_frame', self.curframe) diff --git a/Lib/test/test_gdb/util.py b/Lib/test/test_gdb/util.py index 8fe9cfc543395e..0195e53c4d44e1 100644 --- a/Lib/test/test_gdb/util.py +++ b/Lib/test/test_gdb/util.py @@ -246,7 +246,7 @@ def get_stack_trace(self, source=None, script=None, print(line, file=sys.stderr) # bpo-34007: Sometimes some versions of the shared libraries that - # are part of the traceback are compiled in optimised mode and the + # are part of the traceback are compiled in optimized mode and the # Program Counter (PC) is not present, not allowing gdb to walk the # frames back. When this happens, the Python bindings of gdb raise # an exception, making the test impossible to succeed. diff --git a/Misc/HISTORY b/Misc/HISTORY index b66413277259dc..e489f27d18da6d 100644 --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -15056,7 +15056,7 @@ Tests ----- - Issue #5354: New test support function import_fresh_module() makes - it easy to import both normal and optimised versions of modules. + it easy to import both normal and optimized versions of modules. test_heapq and test_warnings have been adjusted to use it, tests for other modules with both C and Python implementations in the stdlib can be adjusted to use it over time. diff --git a/Misc/NEWS.d/3.11.0a1.rst b/Misc/NEWS.d/3.11.0a1.rst index 40fbb9d42b7944..583715331a4b33 100644 --- a/Misc/NEWS.d/3.11.0a1.rst +++ b/Misc/NEWS.d/3.11.0a1.rst @@ -91,7 +91,7 @@ singleton is destroyed by mistake. Patch by Victor Stinner. .. nonce: ZFgFSj .. section: Core and Builtins -sum() was further optimised for summing up single digit integers. +sum() was further optimized for summing up single digit integers. .. From 672458fa1813c9f12370d9d87224c2d1ae2c4547 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 21 May 2024 12:28:47 +1000 Subject: [PATCH 14/18] Clarify that most optimizations are universal --- Doc/glossary.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index a586145a0c5554..42e2a6f4e301b6 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -894,7 +894,9 @@ Glossary compiler when the code is compiled, allowing optimization of read and write access to these names. The local namespaces for functions, generators, coroutines, comprehensions, and generator expressions are - optimized in this fashion. + optimized in this fashion. Note: most interpreter optimizations are + applied to all scopes, only those relying on a known set of local + and nonlocal variable names are restricted to optimized scopes. package A Python :term:`module` which can contain submodules or recursively, From 270dfaccd1afdc6e93d570aea83b0c21f3f182f5 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 21 May 2024 12:35:54 +1000 Subject: [PATCH 15/18] Fix code.py wording in docs, not just docstring --- Doc/library/code.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/code.rst b/Doc/library/code.rst index 46d5e4807b05ee..8f7692df9fb22d 100644 --- a/Doc/library/code.rst +++ b/Doc/library/code.rst @@ -18,8 +18,8 @@ build applications which provide an interactive interpreter prompt. This class deals with parsing and interpreter state (the user's namespace); it does not deal with input buffering or prompting or input file naming (the filename is always passed in explicitly). The optional *locals* argument - specifies the mapping namespace in which code will be executed; it defaults - to a newly created dictionary with key ``'__name__'`` set to + specifies a mapping to use as the namespace in which code will be executed; + it defaults to a newly created dictionary with key ``'__name__'`` set to ``'__console__'`` and key ``'__doc__'`` set to ``None``. From e0ba6c73125fdb4083fe382d27c8b82dccb63960 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 21 May 2024 12:52:59 +1000 Subject: [PATCH 16/18] exec quirk affects classes, too --- Doc/library/functions.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index c50ef21640a5c0..58e68dce00b1cb 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -620,9 +620,9 @@ are always available. They are listed here in alphabetical order. When ``exec`` gets two separate objects as *globals* and *locals*, the code will be executed as if it were embedded in a class definition. This - means functions defined in the executed code will not be able to access - variables assigned at the top level (as the "top level" variables are - treated as class variables in a class definition). + means functions and classes defined in the executed code will not be able + to access variables assigned at the top level (as the "top level" + variables are treated as class variables in a class definition). Passing a :class:`collections.ChainMap` instance as *globals* allows name lookups to be chained across multiple mappings without triggering this behaviour. Values assigned to top-level names in the executed code can be From 1187f62c201198a806f077f9e2991b7002a5319b Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 21 May 2024 12:58:04 +1000 Subject: [PATCH 17/18] Be consistent about 'top level' --- Doc/library/functions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 58e68dce00b1cb..a879ddbca92e82 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -625,7 +625,7 @@ are always available. They are listed here in alphabetical order. variables are treated as class variables in a class definition). Passing a :class:`collections.ChainMap` instance as *globals* allows name lookups to be chained across multiple mappings without triggering this - behaviour. Values assigned to top-level names in the executed code can be + behaviour. Values assigned to top level names in the executed code can be retrieved by passing an empty dictionary as the first entry in the chain. If the *globals* dictionary does not contain a value for the key From 9f86cf5803df998756027483d91c479b5bbe83a8 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 21 May 2024 13:09:15 +1000 Subject: [PATCH 18/18] Avoid touching unrelated files --- Lib/test/test_gdb/util.py | 2 +- Misc/HISTORY | 2 +- Misc/NEWS.d/3.11.0a1.rst | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_gdb/util.py b/Lib/test/test_gdb/util.py index 0195e53c4d44e1..8fe9cfc543395e 100644 --- a/Lib/test/test_gdb/util.py +++ b/Lib/test/test_gdb/util.py @@ -246,7 +246,7 @@ def get_stack_trace(self, source=None, script=None, print(line, file=sys.stderr) # bpo-34007: Sometimes some versions of the shared libraries that - # are part of the traceback are compiled in optimized mode and the + # are part of the traceback are compiled in optimised mode and the # Program Counter (PC) is not present, not allowing gdb to walk the # frames back. When this happens, the Python bindings of gdb raise # an exception, making the test impossible to succeed. diff --git a/Misc/HISTORY b/Misc/HISTORY index e489f27d18da6d..b66413277259dc 100644 --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -15056,7 +15056,7 @@ Tests ----- - Issue #5354: New test support function import_fresh_module() makes - it easy to import both normal and optimized versions of modules. + it easy to import both normal and optimised versions of modules. test_heapq and test_warnings have been adjusted to use it, tests for other modules with both C and Python implementations in the stdlib can be adjusted to use it over time. diff --git a/Misc/NEWS.d/3.11.0a1.rst b/Misc/NEWS.d/3.11.0a1.rst index 583715331a4b33..40fbb9d42b7944 100644 --- a/Misc/NEWS.d/3.11.0a1.rst +++ b/Misc/NEWS.d/3.11.0a1.rst @@ -91,7 +91,7 @@ singleton is destroyed by mistake. Patch by Victor Stinner. .. nonce: ZFgFSj .. section: Core and Builtins -sum() was further optimized for summing up single digit integers. +sum() was further optimised for summing up single digit integers. ..