While there are some other related issues that could help mitigate the pain of long test discovery runtimes (#23047, #4586), I'm writing this issue to ask about potential opportunities to simply improve the performance of the vscode_pytest.

I'm actually not entirely sure where the bottleneck is. Is it in build_test_tree as it iterates over large numbers of tests, or is it in send_message due to the size of the payload? Either way, the performance I'm seeing is worse than I would expect.

Currently
I have ~150,000 test cases, many of which are marked skip or xfail. Manually running python -m pytest --co -q takes 7s, but test discovery executed by vscode takes over 50s.

At some point in the discovery process, the IDE does seem to hang a little bit (not for the entire 50s, but perhaps for rendering the Test Explorer nodes once the message is tests payload is received?).

Expected

I would expect that building the VSCode test tree could occur in an amount of time that's closer to the test collection time when vscode_python plugin isn't involved. The overhead of building the test tree is greater than seems necessary (without an intimate knowledge of everything it's doing).

Workarounds

I know that I can reduce the frequency of test discovery events and that #4586 may also eventually help with this.