Description
Hi; I'm the author of a game engine which makes heavy use of Python. The engine sends various network requests in background threads for various purposes. Recently I switched these to use https instead of http, and soon after noticed my Android builds in particular started hitching badly.
I tracked this down to my main logic thread occasionally spending upwards of 1 second waiting for a GIL lock, and further tracked that down to SSLContext.set_default_verify_paths(). That gets called by the urllib requests I'm firing off in my bg thread and winds up starving all other threads as it holds on to the GIL for the entirety of the SSL_CTX_set_default_verify_paths() call, which for some reason is taking quite a while on some Android devices.
I can separately look into why the underlying call is taking so long in the Android case, but regardless I'm able to completely eliminate the hitches by releasing the GIL for that call (enclosing the SSL_CTX_set_default_verify_paths() call with PySSL_BEGIN_ALLOW_THREADS/PySSL_END_ALLOW_THREADS).
Is that safe and reasonable to do for that call? I'd be happy to make a PR if so.
This was all tested on Python 3.10.5.
Thanks
-Eric
Lines 4301 to 4310 in 760b8cf