Daemon crash caused by importlib.util missing from import_map #20209
Description
Crash Report
I was investigating #9366 (good news: looks like the original problem there is resolved, but I'm not sure yet) and observed that the pytest repro in #9366 (comment) is now crashing for different reason.
Traceback
Traceback (most recent call last):
File "/home/stas/Documents/Work/mypy/mypy/dmypy/client.py", line 281, in main
args.action(args)
File "/home/stas/Documents/Work/mypy/mypy/dmypy/client.py", line 639, in do_daemon
Server(options, args.status_file, timeout=args.timeout).serve()
File "/home/stas/Documents/Work/mypy/mypy/dmypy_server.py", line 237, in serve
resp = self.run_command(command, data)
File "/home/stas/Documents/Work/mypy/mypy/dmypy_server.py", line 286, in run_command
ret = method(self, **data)
File "/home/stas/Documents/Work/mypy/mypy/dmypy_server.py", line 354, in cmd_run
return self.check(sources, export_types, is_tty, terminal_width)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/dmypy_server.py", line 433, in check
messages = self.fine_grained_increment_follow_imports(
sources, explicit_export_types=export_types
)
File "/home/stas/Documents/Work/mypy/mypy/dmypy_server.py", line 683, in fine_grained_increment_follow_imports
messages = fine_grained_manager.update(new_unsuppressed, [], followed=True)
File "/home/stas/Documents/Work/mypy/mypy/server/update.py", line 265, in update
result = self.update_one(
changed_modules, initial_set, removed_set, blocking_error, followed
)
File "/home/stas/Documents/Work/mypy/mypy/server/update.py", line 367, in update_one
result = self.update_module(next_id, next_path, next_id in removed_set, followed)
File "/home/stas/Documents/Work/mypy/mypy/server/update.py", line 432, in update_module
result = update_module_isolated(
module, path, manager, previous_modules, graph, force_removed, followed
)
File "/home/stas/Documents/Work/mypy/mypy/server/update.py", line 655, in update_module_isolated
semantic_analysis_for_scc(graph, [state.id], manager.errors)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/semanal_main.py", line 96, in semantic_analysis_for_scc
process_functions(graph, scc, patches)
File "/home/stas/Documents/Work/mypy/mypy/semanal_main.py", line 305, in process_functions
process_top_level_function(
File "/home/stas/Documents/Work/mypy/mypy/semanal_main.py", line 344, in process_top_level_function
deferred, incomplete, progress = semantic_analyze_target(
File "/home/stas/Documents/Work/mypy/mypy/semanal_main.py", line 410, in semantic_analyze_target
analyzer.refresh_partial(
File "/home/stas/Documents/Work/mypy/mypy/semanal.py", line 704, in refresh_partial
self.accept(node)
~~~~~~~~~~~^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/semanal.py", line 7566, in accept
report_internal_error(err, self.errors.file, node.line, self.errors, self.options)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/errors.py", line 1334, in report_internal_error
raise err
File "/home/stas/Documents/Work/mypy/mypy/semanal.py", line 7564, in accept
node.accept(self)
~~~~~~~~~~~^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/nodes.py", line 994, in accept
return visitor.visit_func_def(self)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/semanal.py", line 978, in visit_func_def
self.analyze_func_def(defn)
~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/semanal.py", line 994, in analyze_func_def
has_self_type = self.update_function_type_variables(defn.type, defn)
File "/home/stas/Documents/Work/mypy/mypy/semanal.py", line 1216, in update_function_type_variables
fun_type.variables, has_self_type = a.bind_function_type_variables(fun_type, defn)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/typeanal.py", line 1858, in bind_function_type_variables
typevars, has_self_type = self.infer_type_variables(fun_type)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/typeanal.py", line 1836, in infer_type_variables
type.ret_type.accept(visitor)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/types.py", line 3404, in accept
return visitor.visit_union_type(self)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/typeanal.py", line 2667, in visit_union_type
self.process_types(t.items)
~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/typeanal.py", line 2689, in process_types
t.accept(self)
~~~~~~~~^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/types.py", line 1095, in accept
return visitor.visit_unbound_type(self)
~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/typeanal.py", line 2583, in visit_unbound_type
node = self.api.lookup_qualified(name, t)
File "/home/stas/Documents/Work/mypy/mypy/semanal.py", line 6650, in lookup_qualified
nextsym = self.get_module_symbol(node, part)
File "/home/stas/Documents/Work/mypy/mypy/semanal.py", line 6702, in get_module_symbol
if fullname in self.modules and self.is_visible_import(module, fullname):
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/home/stas/Documents/Work/mypy/mypy/semanal.py", line 6722, in is_visible_import
if id in self.import_map[self.cur_mod_id]:
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: 'importlib.util'To Reproduce
This is reproducible in a testcase, but I haven't been able to reproduce without importlib yet. The big pytest-based repro crashes missing _pytest.assertion.rewrite next time if I ignore importlib.util.
[case testDaemonImportMapRefresh]
$ dmypy start
Daemon started
$ dmypy check test.py
Success: no issues found in 1 source file
$ {python} -c 'print("import importlib.util")' > test.py
$ dmypy check test.py
Success: no issues found in 1 source file
[file test.py]
Rebuilding manager.import_map in mypy.server.update::ensure_trees_loaded unconditionally helps (for id in graph: manager.import_map[id] = set(graph[id].dependencies + graph[id].suppressed)), adding that makes the original repro in #9366 pass without crashes or false positive diagnostics, but I hope there's a saner way than that?
Your Environment
- Mypy version used: current master
- Mypy command-line flags: none
- Mypy configuration options from
mypy.ini(and other config files): none - Python version used: 3.13
- Operating system and version: Ubuntu 24.04 LTS, probably irrelevant