Description
Bug report
Bug description:
There is a race condition when closing an asyncio.Server. Accepting a connection takes several iterations of the event loop to complete, and a call to Server.close in the middle will lead to exceptions.
First, some demonstration code. Run both of these concurrently, running the server with python -X dev:
Client:
#!/usr/bin/env python3
import socket
while True:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
try:
sock.connect(("127.0.0.1", 4445))
except IOError:
passServer:
#!/usr/bin/env python3
import asyncio
def cb(reader, writer):
writer.close()
def _my_attach(self):
if self._sockets is None:
print(self._sockets, self._active_count, flush=True)
_orig_attach(self)
async def main():
while True:
server = await asyncio.start_server(cb, host="127.0.0.1", port=4445)
await asyncio.sleep(0)
server.close()
await server.wait_closed()
asyncio.run(main())It will repeatly output errors like this:
Error on transport creation for incoming connection
handle_traceback: Handle created at (most recent call last):
File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
return runner.run(main)
File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
File "/usr/lib/python3.12/asyncio/base_events.py", line 651, in run_until_complete
self.run_forever()
File "/usr/lib/python3.12/asyncio/base_events.py", line 618, in run_forever
self._run_once()
File "/usr/lib/python3.12/asyncio/base_events.py", line 1943, in _run_once
handle._run()
File "/usr/lib/python3.12/asyncio/events.py", line 84, in _run
self._context.run(self._callback, *self._args)
File "/usr/lib/python3.12/asyncio/selector_events.py", line 211, in _accept_connection
self.create_task(accept)
File "/usr/lib/python3.12/asyncio/base_events.py", line 436, in create_task
task = tasks.Task(coro, loop=self, name=name, context=context)
protocol: <asyncio.streams.StreamReaderProtocol object at 0x7f7bb340b350>
Traceback (most recent call last):
File "/usr/lib/python3.12/asyncio/selector_events.py", line 230, in _accept_connection2
transport = self._make_socket_transport(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/selector_events.py", line 72, in _make_socket_transport
return _SelectorSocketTransport(self, sock, protocol, waiter,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/selector_events.py", line 936, in __init__
super().__init__(loop, sock, protocol, extra, server)
File "/usr/lib/python3.12/asyncio/selector_events.py", line 800, in __init__
self._server._attach()
File "/usr/lib/python3.12/asyncio/base_events.py", line 294, in _attach
assert self._sockets is not None
AssertionError
Additionally, stopping the server (with Ctrl-C) prints lots of ResourceWarnings about unclosed transports and sockets.
When a connection arrives on a socket (with selector_events), it goes through the following steps before reaching Server._attach:
- The socket is accepted: here
_accept_connection2is scheduled (will only run on next event loop iteration): here- The transport is created: here
Server._attachis called: here
It's possible for server.close() to be executed between steps 2 and 3, in which case Server._sockets has already been set to None, and the assertion is triggered. The client will presumably experience this as a connection that either closes immediately (if the garbage collector closes the socket) or is unresponsive.
This particular bug might be fixable by creating the transport synchronously in _accept_connection. I think there may be further potential race conditions because connection_made is also called asynchronously, which means it is possible for it to be called after Server.close has already returned. That might not break anything in asyncio itself but it will cause software that does something like the following to hang in 3.12 (related to #79033 / #104344):
server.close()
for (reader, writer) in my_connections:
writer.close()
await writer.closed()
await server.wait_closed()CPython versions tested on:
3.11, 3.12
Operating systems tested on:
Linux
Linked PRs
Metadata
Metadata
Assignees
Labels
Projects
Status