Race condition when importing collections.abc #125245
Description
Bug report
Bug description:
Discovered alongside #125243 with similar steps to reproduce. I don't have a simpler way to trigger this than "run the PyO3 tests in a loop" because I think it requires many threads accessing the python runtime simulatenously.
To trigger it, have a rust toolchain and Python installed, clone the PyO3 repo and execute the following command:
while RUST_BACKTRACE=1 UNSAFE_PYO3_BUILD_FREE_THREADED=1 cargo test --lib --features=full -- --test-threads=100; do :; done
You may also hit some other test failures related to ZoneInfo, see the other issue I opened about that.
You will eventually see a test failure with the following text:
---- exceptions::socket::tests::gaierror stdout ----
thread 'exceptions::socket::tests::gaierror' panicked at src/impl_/exceptions.rs:26:17:
failed to import exception socket.gaierror: ImportError: cannot import name 'Mapping' from 'collections.abc' (/Users/goldbaum/.pyenv/versions/3.13.0t/lib/python3.13t/collections/abc.py)
I slightly modified PyO3 to get a traceback as well (hidden because it's a distractingly long diff):
diff --git a/src/impl_/exceptions.rs b/src/impl_/exceptions.rs
index 15b6f53b..de63ad59 100644
--- a/src/impl_/exceptions.rs
+++ b/src/impl_/exceptions.rs
@@ -1,4 +1,8 @@
-use crate::{sync::GILOnceCell, types::PyType, Bound, Py, Python};
+use crate::{
+ sync::GILOnceCell,
+ types::{PyTracebackMethods, PyType},
+ Bound, Py, Python,
+};
pub struct ImportedExceptionTypeObject {
imported_value: GILOnceCell<Py<PyType>>,
@@ -20,8 +24,11 @@ impl ImportedExceptionTypeObject {
.import(py, self.module, self.name)
.unwrap_or_else(|e| {
panic!(
- "failed to import exception {}.{}: {}",
- self.module, self.name, e
+ "failed to import exception {}.{}: {}\n{}",
+ self.module,
+ self.name,
+ e,
+ e.traceback(py).unwrap().format().unwrap(),
)
})
}
And the traceback is:
Traceback (most recent call last):
File "/Users/goldbaum/.pyenv/versions/3.13.0t/lib/python3.13t/socket.py", line 55, in <module>
import os, sys, io, selectors
File "/Users/goldbaum/.pyenv/versions/3.13.0t/lib/python3.13t/selectors.py", line 10, in <module>
from collections.abc import Mapping
So somehow during setup of the socket module, Mapping isn't available yet, but only if many threads are simultaneously touching the Python runtime.
(ping @davidhewitt, we probably want to disable the socket error tests on the free-threaded build as well)
CPython versions tested on:
3.13
Operating systems tested on:
macOS