Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

test_runner: improve enqueue and dequeue#49751

Closed
MoLow wants to merge 3 commits into
nodejs:mainnodejs/node:mainfrom
MoLow:test-runner-enqueu-dequeueMoLow/node:test-runner-enqueu-dequeueCopy head branch name to clipboard
Closed

test_runner: improve enqueue and dequeue#49751
MoLow wants to merge 3 commits into
nodejs:mainnodejs/node:mainfrom
MoLow:test-runner-enqueu-dequeueMoLow/node:test-runner-enqueu-dequeueCopy head branch name to clipboard

Conversation

@MoLow

@MoLow MoLow commented Sep 21, 2023

Copy link
Copy Markdown
Member

extracting parts out of #48932 to land first, will make it easier for me to complete that big PR

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Sep 21, 2023
@MoLow

MoLow commented Sep 21, 2023

Copy link
Copy Markdown
Member Author

@MoLow MoLow marked this pull request as ready for review September 21, 2023 19:17
@rluvaton

Copy link
Copy Markdown
Member

Benchmark output:

                                                                                                   confidence improvement accuracy (*)   (**)   (***)
test_runner/global-concurrent-tests.js type='async' n=100                                                  **     -5.62 %       ±3.69% ±4.92%  ±6.40%
test_runner/global-concurrent-tests.js type='async' n=1000                                                        -3.01 %       ±6.33% ±8.43% ±10.97%
test_runner/global-concurrent-tests.js type='async' n=10000                                               ***    -11.68 %       ±1.86% ±2.48%  ±3.24%
test_runner/global-concurrent-tests.js type='sync' n=100                                                   **     -8.29 %       ±5.45% ±7.25%  ±9.46%
test_runner/global-concurrent-tests.js type='sync' n=1000                                                         -3.87 %       ±5.29% ±7.04%  ±9.17%
test_runner/global-concurrent-tests.js type='sync' n=10000                                                ***    -12.91 %       ±1.85% ±2.46%  ±3.21%
test_runner/global-sequential-tests.js type='async' n=100                                                         -0.55 %       ±1.62% ±2.16%  ±2.81%
test_runner/global-sequential-tests.js type='async' n=1000                                                         0.71 %       ±2.49% ±3.32%  ±4.33%
test_runner/global-sequential-tests.js type='async' n=10000                                                        0.68 %       ±1.50% ±2.00%  ±2.62%
test_runner/global-sequential-tests.js type='sync' n=100                                                           0.01 %       ±1.30% ±1.74%  ±2.26%
test_runner/global-sequential-tests.js type='sync' n=1000                                                          1.20 %       ±1.40% ±1.86%  ±2.42%
test_runner/global-sequential-tests.js type='sync' n=10000                                                         1.13 %       ±2.15% ±2.88%  ±3.77%
test_runner/multi-files.js concurrency='no' patterns='test/fixtures/test-runner/**/*.?(c|m)js'                     0.02 %       ±0.64% ±0.85%  ±1.11%
test_runner/multi-files.js concurrency='no' patterns='test/parallel/test-runner-*'                                -0.37 %       ±0.99% ±1.32%  ±1.72%
test_runner/multi-files.js concurrency='yes' patterns='test/fixtures/test-runner/**/*.?(c|m)js'                   -0.06 %       ±0.45% ±0.60%  ±0.78%
test_runner/multi-files.js concurrency='yes' patterns='test/parallel/test-runner-*'                                0.80 %       ±4.69% ±6.29%  ±8.29%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=10 numberOfSuites=10           ***    -16.52 %       ±7.34% ±9.79% ±12.79%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=10 numberOfSuites=100                  -5.35 %       ±6.64% ±8.83% ±11.49%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=100 numberOfSuites=10            *     -7.93 %       ±6.35% ±8.46% ±11.03%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=100 numberOfSuites=100         ***    -15.74 %       ±3.83% ±5.12%  ±6.69%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=10 numberOfSuites=10             **     -9.80 %       ±5.75% ±7.66%  ±9.97%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=10 numberOfSuites=100                   -6.93 %       ±7.02% ±9.34% ±12.16%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=100 numberOfSuites=10            **    -10.70 %       ±6.46% ±8.62% ±11.27%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=100 numberOfSuites=100          ***    -15.67 %       ±4.25% ±5.67%  ±7.40%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=10 numberOfSuites=10           **     -9.35 %       ±5.73% ±7.62%  ±9.92%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=10 numberOfSuites=100                 -3.61 %       ±7.13% ±9.49% ±12.35%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=100 numberOfSuites=10                 -5.42 %       ±6.65% ±8.86% ±11.53%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=100 numberOfSuites=100        ***    -15.05 %       ±4.11% ±5.48%  ±7.14%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=10 numberOfSuites=10           ***    -15.59 %       ±6.39% ±8.52% ±11.13%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=10 numberOfSuites=100                  -3.53 %       ±6.85% ±9.12% ±11.87%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=100 numberOfSuites=10                  -2.89 %       ±3.45% ±4.63%  ±6.11%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=100 numberOfSuites=100         ***    -14.85 %       ±4.63% ±6.17%  ±8.04%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 32 comparisons, you can thus
expect the following amount of false-positive results:
  1.60 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.32 false positives, when considering a   1% risk acceptance (**, ***),
  0.03 false positives, when considering a 0.1% risk acceptance (***)

Comment thread benchmark/test_runner/suite-tests.js Outdated
Comment thread benchmark/test_runner/suite-tests.js
Comment thread test/fixtures/test-runner/output/describe_it.js
Comment thread test/fixtures/test-runner/output/describe_it.snapshot
@RelxOff

RelxOff commented Sep 22, 2023

Copy link
Copy Markdown

💯

@MoLow MoLow force-pushed the test-runner-enqueu-dequeue branch from 6626cd4 to 37e1373 Compare September 23, 2023 20:16
@MoLow MoLow closed this Mar 17, 2024
@MoLow MoLow deleted the test-runner-enqueu-dequeue branch May 24, 2024 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.