Feature or enhancement

We propose adding “eager” coroutine execution support to asyncio.TaskGroup via a new method enqueue() [1].

TaskGroup.enqueue() would have the same signature as TaskGroup.create_task() but eagerly perform the first step of the passed coroutine’s execution immediately. If the coroutine completes without yielding, the result of enqueue() would be an object which behaves like a completed asyncio.Task. Otherwise, enqueue() behaves the same as TaskGroup.create_task(), returning a pending asyncio.Task.

The reason for a new method, rather than changing the implementation of TaskGroup.create_task() is this new method introduces a small semantic difference. For example in:

async def coro(): ...

async with TaskGroup() as tg:
  tg.enqueue(coro())
  raise Exception

The exception will cancel everthing scheduled in tg, but if some or all of coro() completes eagerly any side-effects of this will be observable in further execution. If tg.create_task() is used instead no part of coro() will be executed.

Pitch

At Instagram we’ve observed ~70% of coroutine instances passed to asyncio.gather() can run fully synchronously i.e. without performing any I/O which would suspend execution. This typically happens when there is a local cache which can elide actual I/O. We exploit this in Cinder with a modified asyncio.gather() that eagerly executes coroutine args and skips scheduling a asyncio.Task object to an event loop if no yield occurs. Overall this optimization saved ~4% CPU on our Django webservers.

In a prototype implementation of this proposed feature [2] the overhead when scheduling TaskGroups with all fully-synchronous coroutines was decreased by ~8x. When scheduling a mixture of synchronous and asynchronous coroutines, performance is improved by ~1.4x, and when no coroutines can complete synchronously there is still a small improvement.

We anticipate code relying on any semantics which change between TaskGroup.create_task() and TaskGroup.enqueue() will be rare. So, as the TaskGroup interface is new in 3.11, we hope enqueue() and its performance benefits can be promoted as the preferred method for scheduling coroutines in 3.12+.

Previous discussion

This new API was discussed informally at PyCon 2022, with at least some of this being between @gvanrossum, @DinoV, @markshannon, and /or @jbower-fb.

[1] The name "enqueue" came out of a discussion between @gvanrossum and @DinoV.

[2] Prototype implementation (some features missing, e.g. specifying Context), and benchmark.

Linked PRs

• gh-97696: DRAFT asyncio eager tasks factory prototype #101613
• gh-97696: asyncio eager tasks factory #102853
• gh-97696 Remove unnecessary check for eager_start kwarg #104188
• gh-97696 Add documentation for get_coro() behavior with eager tasks #104189
• gh-97696: Remove redundant #include #104216
• gh-97696: Use PyObject_CallMethodNoArgs and inline is_loop_running check #104255
• gh-97696: Improve and fix documentation for asyncio eager tasks #104256
• gh-97696: Move around and update the whatsnew entry for asyncio eager task factory #104298
• gh-97696 Add documentation for get_coro() behavior with eager tasks #104304