gh-83069: Use efficient event-driven `subprocess.Popen.wait()` on Linux / macOS / BSD#144047

Merged

vstinner merged 68 commits into

python:mainfrom

giampaolo:subprocess-fast-wait

Jan 28, 2026

Commits on Jan 18, 2026

Use pidfd_open() and save the FD as an attribute
giampaolo
committed
288e4db
Copy full SHA for 288e4db
Create _busy_wait() and move code in there
giampaolo
committed
6e494ca
Copy full SHA for 6e494ca
Move existing code in new _blocking_wait() method
giampaolo
committed
aaf193c
Copy full SHA for aaf193c
Use _wait_pidfd()
giampaolo
committed
7cbb0ad
Copy full SHA for 7cbb0ad
Use pidfd_open() not at class level but at method level
giampaolo
committed
d9760c3
Copy full SHA for d9760c3
Add kqueue() implementation for macOS and BSD
giampaolo
committed
5fd8eec
Copy full SHA for 5fd8eec
Add docstrings
giampaolo
committed
100b111
Copy full SHA for 100b111
Be conservative and check for specific error codes
giampaolo
committed
74ac2f4
Copy full SHA for 74ac2f4
Document possible failures
giampaolo
committed
fc0cfd6
Copy full SHA for fc0cfd6
Add missing import
giampaolo
committed
f282459
Copy full SHA for f282459
Write test for pidfd_open() failing
giampaolo
committed
6125976
Copy full SHA for 6125976
Write test case for pidfd_open() / kqueue failing. Assert fallback is used.
giampaolo
committed
41dc6c0
Copy full SHA for 41dc6c0
Write test case for kqueue failing. Assert fallback is used.
giampaolo
committed
73ba380
Copy full SHA for 73ba380
Move tests in their own class
giampaolo
committed
3c8e603
Copy full SHA for 3c8e603
Add test for terminated PID race
giampaolo
committed
932ae58
Copy full SHA for 932ae58

Commits on Jan 19, 2026

Add test_kqueue_race()
giampaolo
committed
bb5080a
Copy full SHA for bb5080a
Add test_kqueue_control_error
giampaolo
committed
f067073
Copy full SHA for f067073
Add docstring
giampaolo
committed
4d96c2f
Copy full SHA for 4d96c2f
Guard against possible slow test
giampaolo
committed
fe05acc
Copy full SHA for fe05acc
Timeout: use math.ceil to avoid truncation
giampaolo
committed
adb444e
Copy full SHA for adb444e
Remove unused exception var
giampaolo
committed
4ec17c1
Copy full SHA for 4ec17c1
Timeout: use math.ceil to avoid truncation
giampaolo
committed
e807ba9
Copy full SHA for e807ba9
Replace _can_use_kqueue() -> _CAN_USE_KQUEUE
giampaolo
committed
1ddc52b
Copy full SHA for 1ddc52b
Shorten code
giampaolo
committed
645ef6c
Copy full SHA for 645ef6c
Use waitpid() + WNOHANG even if process exited to avoid rare race
giampaolo
committed
6ee771b
Copy full SHA for 6ee771b
Revert prev change
giampaolo
committed
61c6b99
Copy full SHA for 61c6b99
Use ceil(timeout) to avoid truncation
giampaolo
committed
527646d
Copy full SHA for 527646d
Remove check for timeout < 0 + rm test case which didn't make sense.
giampaolo
committed
0a8a1b2
Copy full SHA for 0a8a1b2
Don't use _blocking_wait() as it has a while loop
giampaolo
committed
73b97dc
Copy full SHA for 73b97dc
Add docstring
giampaolo
committed
2d3c3f7
Copy full SHA for 2d3c3f7
Rm _busy_wait()
giampaolo
committed
4eac42f
Copy full SHA for 4eac42f
Add comment
giampaolo
committed
3c156a9
Copy full SHA for 3c156a9
Add assert
giampaolo
committed
dac7d3b
Copy full SHA for dac7d3b
Update comments
giampaolo
committed
5c7ec2f
Copy full SHA for 5c7ec2f
Add test for timeout=0
giampaolo
committed
5c29144
Copy full SHA for 5c29144
Update comment about PID reuse race
giampaolo
committed
4359b07
Copy full SHA for 4359b07
Update comment
giampaolo
committed
81275c8
Copy full SHA for 81275c8
Handle rare case where poll() says we're done, but waitpid() doesn't
giampaolo
committed
43b500f
Copy full SHA for 43b500f
Update Doc/library/subprocess.rst
giampaolo
committed
b64e42b
Copy full SHA for b64e42b
Add news entry
giampaolo
committed
a101406
Copy full SHA for a101406
Add entry in Doc/whatsnew/3.15.rst
giampaolo
committed
452f8c4
Copy full SHA for 452f8c4
Fix typo
giampaolo
committed
b0c9890
Copy full SHA for b0c9890
Re-wording
giampaolo
committed
27b7c9f
Copy full SHA for 27b7c9f
Raise on timeout < 0 and re-add test case
giampaolo
committed
e1da996
Copy full SHA for e1da996
Check if can really use can_use_pidfd() in unit tests
giampaolo
committed
5c78acc
Copy full SHA for 5c78acc
Check if can really use kqueue() in unit tests
giampaolo
committed
df0538a
Copy full SHA for df0538a
Pre-emptively check whether to use the fast way methods
giampaolo
committed
6d8e36c
Copy full SHA for 6d8e36c
Add test_fast_path_avoid_busy_loop
giampaolo
committed
6ba7465
Copy full SHA for 6ba7465
Update comments
giampaolo
committed
e3c7977
Copy full SHA for e3c7977
Fix missing import on Windows
giampaolo
committed
97cc3be
Copy full SHA for 97cc3be
Merge branch 'main' into subprocess-fast-wait
giampaolo
committed
c4342e3
Copy full SHA for c4342e3
Try to fix doc build error
giampaolo
committed
5d78d24
Copy full SHA for 5d78d24
Try to fix doc build error 2
giampaolo
committed
a916da3
Copy full SHA for a916da3
Try to fix doc build error 3
giampaolo
committed
86200bd
Copy full SHA for 86200bd
Try to fix doc build error 4
giampaolo
committed
85c38bc
Copy full SHA for 85c38bc
Try to fix doc build error 5
giampaolo
committed
3c92c1d
Copy full SHA for 3c92c1d
Minor rewordings
giampaolo
committed
33c8b1f
Copy full SHA for 33c8b1f

Commits on Jan 21, 2026

Small refact
giampaolo
committed
525c047
Copy full SHA for 525c047
Address review comments (bare assert, don't use ceil())
giampaolo
committed
2b931b8
Copy full SHA for 2b931b8
Address review comments (use test class VARIABLES)
giampaolo
committed
d1c6e91
Copy full SHA for d1c6e91
Address review comments (add periods in doc)
giampaolo
committed
3e9d303
Copy full SHA for 3e9d303
Merge branch 'main' into subprocess-fast-wait
giampaolo
committed
a44b2d7
Copy full SHA for a44b2d7
Remove weird unicode char from doc added by accident
giampaolo
committed
b4f6020
Copy full SHA for b4f6020
Don't import select on Windows
giampaolo
committed
4e17856
Copy full SHA for 4e17856
Make kq.control() invocation clearer by passing kwargs
giampaolo
committed
6f6600d
Copy full SHA for 6f6600d
Revert previous commit
giampaolo
committed
37288c6
Copy full SHA for 37288c6

Commits on Jan 22, 2026

Remove bare assert statements
giampaolo
committed
300692e
Copy full SHA for 300692e
Merge branch 'main' into subprocess-fast-wait
giampaolo
authored
7bee1e9
Copy full SHA for 7bee1e9