Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 97fa075

Browse filesBrowse files
cjihrigMoLow
authored and
RafaelGSS
committed
test_runner: support running tests in process
This commit introduces a new --experimental-test-isolation flag that, when set to 'none', causes the test runner to execute all tests in the same process. By default, this is the main test runner process, but if watch mode is enabled, it spawns a separate process that runs all of the tests. The default value of the new flag is 'process', which uses the existing behavior of running each test file in its own child process. It is worth noting that when the isolation mode is 'none', globals and all other top level logic (such as top level before() and after() hooks) is shared among all files. Co-authored-by: Moshe Atlow <moshe@atlow.co.il> PR-URL: #53927 Fixes: #51548 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
1 parent 858b583 commit 97fa075
Copy full SHA for 97fa075
Expand file treeCollapse file tree

22 files changed

+746
-206
lines changed
Open diff view settings
Collapse file

‎doc/api/cli.md‎

Copy file name to clipboardExpand all lines: doc/api/cli.md
+19-2Lines changed: 19 additions & 2 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -1081,6 +1081,20 @@ generated as part of the test runner output. If no tests are run, a coverage
10811081
report is not generated. See the documentation on
10821082
[collecting code coverage from tests][] for more details.
10831083

1084+
### `--experimental-test-isolation=mode`
1085+
1086+
<!-- YAML
1087+
added: REPLACEME
1088+
-->
1089+
1090+
> Stability: 1.0 - Early development
1091+
1092+
Configures the type of test isolation used in the test runner. When `mode` is
1093+
`'process'`, each test file is run in a separate child process. When `mode` is
1094+
`'none'`, all test files run in the same process as the test runner. The default
1095+
isolation mode is `'process'`. This flag is ignored if the `--test` flag is not
1096+
present. See the [test runner execution model][] section for more information.
1097+
10841098
### `--experimental-test-module-mocks`
10851099

10861100
<!-- YAML
@@ -2200,7 +2214,9 @@ added:
22002214
-->
22012215

22022216
The maximum number of test files that the test runner CLI will execute
2203-
concurrently. The default value is `os.availableParallelism() - 1`.
2217+
concurrently. If `--experimental-test-isolation` is set to `'none'`, this flag
2218+
is ignored and concurrency is one. Otherwise, concurrency defaults to
2219+
`os.availableParallelism() - 1`.
22042220

22052221
### `--test-coverage-exclude`
22062222

@@ -2363,7 +2379,7 @@ added: v22.3.0
23632379

23642380
> Stability: 1.0 - Early development
23652381
2366-
Regenerates the snapshot file used by the test runner for [snapshot testing][].
2382+
Regenerates the snapshot files used by the test runner for [snapshot testing][].
23672383
Node.js must be started with the `--experimental-test-snapshots` flag in order
23682384
to use this functionality.
23692385

@@ -3573,6 +3589,7 @@ node --stack-trace-limit=12 -p -e "Error.stackTraceLimit" # prints 12
35733589
[snapshot testing]: test.md#snapshot-testing
35743590
[syntax detection]: packages.md#syntax-detection
35753591
[test reporters]: test.md#test-reporters
3592+
[test runner execution model]: test.md#test-runner-execution-model
35763593
[timezone IDs]: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
35773594
[tracking issue for user-land snapshots]: https://github.com/nodejs/node/issues/44014
35783595
[ways that `TZ` is handled in other environments]: https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html
Collapse file

‎doc/api/test.md‎

Copy file name to clipboardExpand all lines: doc/api/test.md
+28-12Lines changed: 28 additions & 12 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -445,18 +445,26 @@ in the [test runner execution model][] section.
445445

446446
### Test runner execution model
447447

448-
Each matching test file is executed in a separate child process. The maximum
449-
number of child processes running at any time is controlled by the
450-
[`--test-concurrency`][] flag. If the child process finishes with an exit code
451-
of 0, the test is considered passing. Otherwise, the test is considered to be a
452-
failure. Test files must be executable by Node.js, but are not required to use
453-
the `node:test` module internally.
448+
When process-level test isolation is enabled, each matching test file is
449+
executed in a separate child process. The maximum number of child processes
450+
running at any time is controlled by the [`--test-concurrency`][] flag. If the
451+
child process finishes with an exit code of 0, the test is considered passing.
452+
Otherwise, the test is considered to be a failure. Test files must be executable
453+
by Node.js, but are not required to use the `node:test` module internally.
454454

455455
Each test file is executed as if it was a regular script. That is, if the test
456456
file itself uses `node:test` to define tests, all of those tests will be
457457
executed within a single application thread, regardless of the value of the
458458
`concurrency` option of [`test()`][].
459459

460+
When process-level test isolation is disabled, each matching test file is
461+
imported into the test runner process. Once all test files have been loaded, the
462+
top level tests are executed with a concurrency of one. Because the test files
463+
are all run within the same context, it is possible for tests to interact with
464+
each other in ways that are not possible when isolation is enabled. For example,
465+
if a test relies on global state, it is possible for that state to be modified
466+
by a test originating from another file.
467+
460468
## Collecting code coverage
461469

462470
> Stability: 1 - Experimental
@@ -933,7 +941,7 @@ the [`--experimental-test-snapshots`][] command-line flag.
933941
Snapshot files are generated by starting Node.js with the
934942
[`--test-update-snapshots`][] command-line flag. A separate snapshot file is
935943
generated for each test file. By default, the snapshot file has the same name
936-
as `process.argv[1]` with a `.snapshot` file extension. This behavior can be
944+
as the test file with a `.snapshot` file extension. This behavior can be
937945
configured using the `snapshot.setResolveSnapshotPath()` function. Each
938946
snapshot assertion corresponds to an export in the snapshot file.
939947

@@ -1239,6 +1247,9 @@ added:
12391247
- v18.9.0
12401248
- v16.19.0
12411249
changes:
1250+
- version: REPLACEME
1251+
pr-url: https://github.com/nodejs/node/pull/53927
1252+
description: Added the `isolation` option.
12421253
- version: v22.6.0
12431254
pr-url: https://github.com/nodejs/node/pull/53866
12441255
description: Added the `globPatterns` option.
@@ -1272,8 +1283,13 @@ changes:
12721283
* `inspectPort` {number|Function} Sets inspector port of test child process.
12731284
This can be a number, or a function that takes no arguments and returns a
12741285
number. If a nullish value is provided, each process gets its own port,
1275-
incremented from the primary's `process.debugPort`.
1276-
**Default:** `undefined`.
1286+
incremented from the primary's `process.debugPort`. This option is ignored
1287+
if the `isolation` option is set to `'none'` as no child processes are
1288+
spawned. **Default:** `undefined`.
1289+
* `isolation` {string} Configures the type of test isolation. If set to
1290+
`'process'`, each test file is run in a separate child process. If set to
1291+
`'none'`, all test files run in the current process. **Default:**
1292+
`'process'`.
12771293
* `only`: {boolean} If truthy, the test context will only run tests that
12781294
have the `only` option set
12791295
* `setup` {Function} A function that accepts the `TestsStream` instance
@@ -1717,9 +1733,9 @@ added: v22.3.0
17171733
17181734
* `fn` {Function} A function used to compute the location of the snapshot file.
17191735
The function receives the path of the test file as its only argument. If the
1720-
`process.argv[1]` is not associated with a file (for example in the REPL),
1721-
the input is undefined. `fn()` must return a string specifying the location of
1722-
the snapshot file.
1736+
test is not associated with a file (for example in the REPL), the input is
1737+
undefined. `fn()` must return a string specifying the location of the snapshot
1738+
snapshot file.
17231739

17241740
This function is used to customize the location of the snapshot file used for
17251741
snapshot testing. By default, the snapshot filename is the same as the entry
Collapse file

‎doc/node.1‎

Copy file name to clipboardExpand all lines: doc/node.1
+3Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,9 @@ Enable the experimental node:sqlite module.
185185
.It Fl -experimental-test-coverage
186186
Enable code coverage in the test runner.
187187
.
188+
.It Fl -experimental-test-isolation Ns = Ns Ar mode
189+
Configures the type of test isolation used in the test runner.
190+
.
188191
.It Fl -experimental-test-module-mocks
189192
Enable module mocking in the test runner.
190193
.
Collapse file

‎lib/internal/main/test_runner.js‎

Copy file name to clipboardExpand all lines: lib/internal/main/test_runner.js
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ markBootstrapComplete();
2121

2222
const options = parseCommandLine();
2323

24-
if (isUsingInspector()) {
24+
if (isUsingInspector() && options.isolation === 'process') {
2525
process.emitWarning('Using the inspector with --test forces running at a concurrency of 1. ' +
2626
'Use the inspectPort option to run with concurrency');
2727
options.concurrency = 1;
Collapse file

‎lib/internal/test_runner/harness.js‎

Copy file name to clipboardExpand all lines: lib/internal/test_runner/harness.js
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -334,4 +334,5 @@ module.exports = {
334334
after: hook('after'),
335335
beforeEach: hook('beforeEach'),
336336
afterEach: hook('afterEach'),
337+
startSubtestAfterBootstrap,
337338
};

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.