Closed
Description
Bug report
Bug description:
Test program:
# An importlib.util.LazyLoader test, based on example from
# https://docs.python.org/3/library/importlib.html#implementing-lazy-imports
import sys
import importlib.util
def lazy_import(name):
spec = importlib.util.find_spec(name)
loader = importlib.util.LazyLoader(spec.loader)
spec.loader = loader
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
loader.exec_module(module)
return module
# Lazy-load the module...
lazy_module = lazy_import(sys.argv[1])
# ... and then trigger load by listing its contents
print(dir(lazy_module))Running under python 3.13.0a4:
$ python3 test_program.py json
['JSONDecodeError', 'JSONDecoder', 'JSONEncoder', '__all__', '__author__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_default_decoder', '_default_encoder', 'codecs', 'decoder', 'detect_encoding', 'dump', 'dumps', 'encoder', 'load', 'loads', 'scanner']$ python3 test_program.py typing
['ABCMeta', 'AbstractSet', 'Annotated', 'Any', 'AnyStr', 'AsyncGenerator', 'AsyncIterable', 'AsyncIterator', 'Awaitable', 'BinaryIO', 'ByteString', 'CT_co', 'Callable', 'ChainMap', 'ClassVar', 'Collection', 'Concatenate', 'Container', 'Coroutine', 'Counter', 'DefaultDict', 'Deque', 'Dict', 'EXCLUDED_ATTRIBUTES', 'Final', 'ForwardRef', 'FrozenSet', 'Generator', 'Generic', 'GenericAlias', 'Hashable', 'IO', 'ItemsView', 'Iterable', 'Iterator', 'KT', 'KeysView', 'List', 'Literal', 'LiteralString', 'Mapping', 'MappingView', 'MethodDescriptorType', 'MethodWrapperType', 'MutableMapping', 'MutableSequence', 'MutableSet', 'NamedTuple', 'NamedTupleMeta', 'Never', 'NewType', 'NoReturn', 'NotRequired', 'Optional', 'OrderedDict', 'ParamSpec', 'ParamSpecArgs', 'ParamSpecKwargs', 'Protocol', 'Required', 'Reversible', 'Self', 'Sequence', 'Set', 'Sized', 'SupportsAbs', 'SupportsBytes', 'SupportsComplex', 'SupportsFloat', 'SupportsIndex', 'SupportsInt', 'SupportsRound', 'T', 'TYPE_CHECKING', 'T_co', 'T_contra', 'Text', 'TextIO', 'Tuple', 'Type', 'TypeAlias', 'TypeAliasType', 'TypeGuard', 'TypeVar', 'TypeVarTuple', 'TypedDict', 'Union', 'Unpack', 'VT', 'VT_co', 'V_co', 'ValuesView', 'WrapperDescriptorType', '_ASSERT_NEVER_REPR_MAX_LENGTH', '_AnnotatedAlias', '_AnyMeta', '_BaseGenericAlias', '_CallableGenericAlias', '_CallableType', '_ConcatenateGenericAlias', '_DeprecatedGenericAlias', '_Final', '_Func', '_GenericAlias', '_IdentityCallable', '_LiteralGenericAlias', '_NamedTuple', '_NotIterable', '_PROTO_ALLOWLIST', '_ProtocolMeta', '_SPECIAL_NAMES', '_Sentinel', '_SpecialForm', '_SpecialGenericAlias', '_TYPING_INTERNALS', '_TupleType', '_TypedCacheSpecialForm', '_TypedDict', '_TypedDictMeta', '_TypingEllipsis', '_UnionGenericAlias', '_UnpackGenericAlias', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__getattr__', '__loader__', '__name__', '__package__', '__spec__', '_abc_instancecheck', '_abc_subclasscheck', '_alias', '_allow_reckless_class_checks', '_allowed_types', '_caches', '_caller', '_check_generic', '_cleanups', '_collect_parameters', '_deduplicate', '_eval_type', '_flatten_literal_params', '_generic_class_getitem', '_generic_init_subclass', '_get_protocol_attrs', '_idfunc', '_is_dunder', '_is_param_expr', '_is_typevar_like', '_is_unpacked_typevartuple', '_lazy_load_getattr_static', '_make_nmtuple', '_make_union', '_namedtuple_mro_entries', '_no_init_or_replace_init', '_overload_dummy', '_overload_registry', '_paramspec_prepare_subst', '_paramspec_subst', '_prohibited', '_proto_hook', '_remove_dups_flatten', '_sentinel', '_should_unflatten_callable_args', '_special', '_strip_annotations', '_tp_cache', '_type_check', '_type_check_issubclass_arg_1', '_type_convert', '_type_repr', '_typevar_subst', '_typevartuple_prepare_subst', '_unpack_args', '_value_and_type_iter', 'abstractmethod', 'assert_never', 'assert_type', 'cast', 'clear_overloads', 'collections', 'copyreg', 'dataclass_transform', 'defaultdict', 'final', 'functools', 'get_args', 'get_origin', 'get_overloads', 'get_protocol_members', 'get_type_hints', 'is_protocol', 'is_typeddict', 'no_type_check', 'no_type_check_decorator', 'operator', 'overload', 'override', 'reveal_type', 'runtime_checkable', 'sys', 'types']Running under python 3.13.0a5:
$ python3 test_program.py json
Traceback (most recent call last):
File "/home/rok/tmp/pyi-py313/test_program.py", line 21, in <module>
print(dir(lazy_module))
~~~^^^^^^^^^^^^^
File "<frozen importlib.util>", line 209, in __getattribute__
File "<frozen importlib._bootstrap_external>", line 1015, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/home/rok/python3.13-bin/lib/python3.13/json/__init__.py", line 106, in <module>
from .decoder import JSONDecoder, JSONDecodeError
File "/home/rok/python3.13-bin/lib/python3.13/json/decoder.py", line 5, in <module>
from json import scanner
File "<frozen importlib.util>", line 209, in __getattribute__
File "/home/rok/python3.13-bin/lib/python3.13/json/__init__.py", line 106, in <module>
from .decoder import JSONDecoder, JSONDecodeError
ImportError: cannot import name 'JSONDecoder' from partially initialized module 'json.decoder' (most likely due to a circular import) (/home/rok/python3.13-bin/lib/python3.13/json/decoder.py)$ python3 test_program.py typing
Traceback (most recent call last):
File "/home/rok/tmp/pyi-py313/test_program.py", line 21, in <module>
print(dir(lazy_module))
~~~^^^^^^^^^^^^^
File "<frozen importlib.util>", line 209, in __getattribute__
File "<frozen importlib._bootstrap_external>", line 1015, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/home/rok/python3.13-bin/lib/python3.13/typing.py", line 1961, in <module>
class Protocol(Generic, metaclass=_ProtocolMeta):
...<49 lines>...
cls.__init__ = _no_init_or_replace_init
File "/home/rok/python3.13-bin/lib/python3.13/typing.py", line 1870, in __new__
return super().__new__(mcls, name, bases, namespace, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen abc>", line 106, in __new__
File "<frozen importlib.util>", line 209, in __getattribute__
File "<frozen importlib._bootstrap_external>", line 1015, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/home/rok/python3.13-bin/lib/python3.13/typing.py", line 1961, in <module>
class Protocol(Generic, metaclass=_ProtocolMeta):
...<49 lines>...
cls.__init__ = _no_init_or_replace_init
File "/home/rok/python3.13-bin/lib/python3.13/typing.py", line 1870, in __new__
return super().__new__(mcls, name, bases, namespace, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[...]
File "/home/rok/python3.13-bin/lib/python3.13/typing.py", line 1870, in __new__
return super().__new__(mcls, name, bases, namespace, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen abc>", line 106, in __new__
File "<frozen importlib.util>", line 209, in __getattribute__
File "<frozen importlib._bootstrap_external>", line 1015, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/home/rok/python3.13-bin/lib/python3.13/typing.py", line 1961, in <module>
class Protocol(Generic, metaclass=_ProtocolMeta):
...<49 lines>...
cls.__init__ = _no_init_or_replace_init
File "/home/rok/python3.13-bin/lib/python3.13/typing.py", line 1870, in __new__
return super().__new__(mcls, name, bases, namespace, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen abc>", line 106, in __new__
File "<frozen importlib.util>", line 209, in __getattribute__
File "<frozen importlib._bootstrap_external>", line 1015, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/home/rok/python3.13-bin/lib/python3.13/typing.py", line 1961, in <module>
class Protocol(Generic, metaclass=_ProtocolMeta):
...<49 lines>...
cls.__init__ = _no_init_or_replace_init
File "/home/rok/python3.13-bin/lib/python3.13/typing.py", line 1870, in __new__
return super().__new__(mcls, name, bases, namespace, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen abc>", line 106, in __new__
File "<frozen importlib.util>", line 209, in __getattribute__
File "<frozen importlib._bootstrap_external>", line 1011, in exec_module
File "<frozen importlib._bootstrap_external>", line 1089, in get_code
RecursionError: maximum recursion depth exceededBisection points at 200271c from #114781. Since this was backported to 3.11 and 3.12 branches, I expect to see the same problem in the next 3.11 and 3.12 releases.
At cursory glance, it seems that moving self.__class__ = types.ModuleType to the very end of loading does not play well with modules/packages whose initialization ends up referring to themselves (e.g., from . import something, or by importing a module that ends up referring to the lazy-loaded module).
CPython versions tested on:
3.13
Operating systems tested on:
Linux, macOS, Windows
Linked PRs
Metadata
Metadata
Assignees
Labels
Projects
Status
Done