Description
Bug report
Bug description:
The following from the docs works:
from multiprocessing import Process, Queue
def f(q):
q.put([42, None, 'hello'])
if __name__ == '__main__':
q = Queue()
p = Process(target=f, args=(q,))
p.start()
print(q.get()) # prints "[42, None, 'hello']"
p.join()The following also works:
from multiprocessing import Process, Queue
from time import sleep
def f(q):
pass
if __name__ == '__main__':
q = Queue()
p = Process(target=f, args=(q,))
p.start()
sleep(1)This works just fine:
class Foo:
def __init__(self, q) -> None:
pass
if __name__ == '__main__':
q = Queue()
p = Process(target=Foo, args=(q,))
p.start()
sleep(1)But the following does NOT work:
from multiprocessing import Process, Queue
from time import sleep
class Foo:
def __init__(self, q) -> None:
pass
class Bar:
def __init__(self) -> None:
q = Queue()
p = Process(target=Foo, args=(q,))
p.start()
if __name__ == '__main__':
Bar()
sleep(1)Which now gives this traceback:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/spawn.py", line 122, in spawn_main
exitcode = _main(fd, parent_sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/spawn.py", line 132, in _main
self = reduction.pickle.load(from_parent)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/synchronize.py", line 115, in __setstate__
self._semlock = _multiprocessing.SemLock._rebuild(*state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory
The following does work, though:
from multiprocessing import Process, Queue
from time import sleep
class Foo:
def __init__(self, q) -> None:
q.put("INIT COMPLETE")
class Bar:
def __init__(self) -> None:
q = Queue()
p = Process(target=Foo, args=(q,))
p.start()
q.get()
if __name__ == '__main__':
Bar()
sleep(1)This should either be documented or fixed, preferably the latter. This behavior is strange and, quite frankly, unexpected. It took me quite a while to figure this out and fix my code but adding an init queue item is not ideal. Note that the following also breaks with the same error:
from multiprocessing import Process, Queue
from time import sleep
def foo(q) -> None:
pass
class Bar:
def __init__(self) -> None:
q = Queue()
p = Process(target=foo, args=(q,))
p.start()
if __name__ == '__main__':
Bar()
sleep(1)Moving the Queue outside of the class (when calling it) also doesn't fix the issue:
from multiprocessing import Process, Queue
from time import sleep
def foo(q) -> None:
pass
class Bar:
def __init__(self, q) -> None:
p = Process(target=foo, args=(q,))
p.start()
if __name__ == '__main__':
Bar(Queue())
sleep(1)But the following works just fine:
from multiprocessing import Process, Queue
from time import sleep
def foo(q) -> None:
pass
class Bar:
def __init__(self, q) -> None:
p = Process(target=foo, args=(q,))
p.start()
if __name__ == '__main__':
q = Queue()
Bar(q)
sleep(1)This may be related to #116526 but I don't personally think this should be classified as the same issue. May be related to #23376 as well but this may still be a separate issue. Neither address this kind of behavior, however.
CPython versions tested on:
3.12
Operating systems tested on:
macOS