Description
Feature or enhancement
We propose adding “eager” coroutine execution support to asyncio.TaskGroup
via a new method enqueue()
[1].
TaskGroup.enqueue()
would have the same signature as TaskGroup.create_task()
but eagerly perform the first step of the passed coroutine
’s execution immediately. If the coroutine
completes without yielding, the result of enqueue()
would be an object which behaves like a completed asyncio.Task
. Otherwise, enqueue()
behaves the same as TaskGroup.create_task()
, returning a pending asyncio.Task
.
The reason for a new method, rather than changing the implementation of TaskGroup.create_task()
is this new method introduces a small semantic difference. For example in:
async def coro(): ...
async with TaskGroup() as tg:
tg.enqueue(coro())
raise Exception
The exception will cancel everthing scheduled in tg
, but if some or all of coro()
completes eagerly any side-effects of this will be observable in further execution. If tg.create_task()
is used instead no part of coro()
will be executed.
Pitch
At Instagram we’ve observed ~70% of coroutine instances passed to asyncio.gather()
can run fully synchronously i.e. without performing any I/O which would suspend execution. This typically happens when there is a local cache which can elide actual I/O. We exploit this in Cinder with a modified asyncio.gather()
that eagerly executes coroutine
args and skips scheduling a asyncio.Task
object to an event loop if no yield occurs. Overall this optimization saved ~4% CPU on our Django webservers.
In a prototype implementation of this proposed feature [2] the overhead when scheduling TaskGroup
s with all fully-synchronous coroutines was decreased by ~8x. When scheduling a mixture of synchronous and asynchronous coroutine
s, performance is improved by ~1.4x, and when no coroutine
s can complete synchronously there is still a small improvement.
We anticipate code relying on any semantics which change between TaskGroup.create_task()
and TaskGroup.enqueue()
will be rare. So, as the TaskGroup interface is new in 3.11, we hope enqueue()
and its performance benefits can be promoted as the preferred method for scheduling coroutines in 3.12+.
Previous discussion
This new API was discussed informally at PyCon 2022, with at least some of this being between @gvanrossum, @DinoV, @markshannon, and /or @jbower-fb.
[1] The name "enqueue
" came out of a discussion between @gvanrossum and @DinoV.
[2] Prototype implementation (some features missing, e.g. specifying Context), and benchmark.
Linked PRs
- gh-97696: DRAFT asyncio eager tasks factory prototype #101613
- gh-97696: asyncio eager tasks factory #102853
- gh-97696 Remove unnecessary check for eager_start kwarg #104188
- gh-97696 Add documentation for get_coro() behavior with eager tasks #104189
- gh-97696: Remove redundant #include #104216
- gh-97696: Use PyObject_CallMethodNoArgs and inline is_loop_running check #104255
- gh-97696: Improve and fix documentation for asyncio eager tasks #104256
- gh-97696: Move around and update the whatsnew entry for asyncio eager task factory #104298
- gh-97696 Add documentation for get_coro() behavior with eager tasks #104304
Metadata
Metadata
Assignees
Labels
Projects
Status
Status