Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit d762a34

Browse filesBrowse files
cjihrigdanielleadams
cjihrig
authored and
danielleadams
committed
test_runner: add --test-name-pattern CLI flag
This commit adds support for running tests that match a regular expression. Fixes: #42984
1 parent e92b074 commit d762a34
Copy full SHA for d762a34

File tree

Expand file treeCollapse file tree

12 files changed

+333
-4
lines changed
Open diff view settings
Filter options
Expand file treeCollapse file tree

12 files changed

+333
-4
lines changed
Open diff view settings
Collapse file

‎doc/api/cli.md‎

Copy file name to clipboardExpand all lines: doc/api/cli.md
+11Lines changed: 11 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -1187,6 +1187,16 @@ Starts the Node.js command line test runner. This flag cannot be combined with
11871187
`--check`, `--eval`, `--interactive`, or the inspector. See the documentation
11881188
on [running tests from the command line][] for more details.
11891189

1190+
### `--test-name-pattern`
1191+
1192+
<!-- YAML
1193+
added: REPLACEME
1194+
-->
1195+
1196+
A regular expression that configures the test runner to only execute tests
1197+
whose name matches the provided pattern. See the documentation on
1198+
[filtering tests by name][] for more details.
1199+
11901200
### `--test-only`
11911201

11921202
<!-- YAML
@@ -2262,6 +2272,7 @@ done
22622272
[debugger]: debugger.md
22632273
[debugging security implications]: https://nodejs.org/en/docs/guides/debugging-getting-started/#security-implications
22642274
[emit_warning]: process.md#processemitwarningwarning-options
2275+
[filtering tests by name]: test.md#filtering-tests-by-name
22652276
[jitless]: https://v8.dev/blog/jitless
22662277
[libuv threadpool documentation]: https://docs.libuv.org/en/latest/threadpool.html
22672278
[remote code execution]: https://www.owasp.org/index.php/Code_Injection
Collapse file

‎doc/api/test.md‎

Copy file name to clipboardExpand all lines: doc/api/test.md
+37Lines changed: 37 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,42 @@ test('this test is not run', () => {
220220
});
221221
```
222222

223+
## Filtering tests by name
224+
225+
The [`--test-name-pattern`][] command-line option can be used to only run tests
226+
whose name matches the provided pattern. Test name patterns are interpreted as
227+
JavaScript regular expressions. The `--test-name-pattern` option can be
228+
specified multiple times in order to run nested tests. For each test that is
229+
executed, any corresponding test hooks, such as `beforeEach()`, are also
230+
run.
231+
232+
Given the following test file, starting Node.js with the
233+
`--test-name-pattern="test [1-3]"` option would cause the test runner to execute
234+
`test 1`, `test 2`, and `test 3`. If `test 1` did not match the test name
235+
pattern, then its subtests would not execute, despite matching the pattern. The
236+
same set of tests could also be executed by passing `--test-name-pattern`
237+
multiple times (e.g. `--test-name-pattern="test 1"`,
238+
`--test-name-pattern="test 2"`, etc.).
239+
240+
```js
241+
test('test 1', async (t) => {
242+
await t.test('test 2');
243+
await t.test('test 3');
244+
});
245+
246+
test('Test 4', async (t) => {
247+
await t.test('Test 5');
248+
await t.test('test 6');
249+
});
250+
```
251+
252+
Test name patterns can also be specified using regular expression literals. This
253+
allows regular expression flags to be used. In the previous example, starting
254+
Node.js with `--test-name-pattern="/test [4-5]/i"` would match `Test 4` and
255+
`Test 5` because the pattern is case-insensitive.
256+
257+
Test name patterns do not change the set of files that the test runner executes.
258+
223259
## Extraneous asynchronous activity
224260

225261
Once a test function finishes executing, the TAP results are output as quickly
@@ -896,6 +932,7 @@ added: v18.7.0
896932
aborted.
897933

898934
[TAP]: https://testanything.org/
935+
[`--test-name-pattern`]: cli.md#--test-name-pattern
899936
[`--test-only`]: cli.md#--test-only
900937
[`--test`]: cli.md#--test
901938
[`SuiteContext`]: #class-suitecontext
Collapse file

‎doc/node.1‎

Copy file name to clipboardExpand all lines: doc/node.1
+4Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -387,6 +387,10 @@ Specify the minimum allocation from the OpenSSL secure heap. The default is 2. T
387387
.It Fl -test
388388
Starts the Node.js command line test runner.
389389
.
390+
.It Fl -test-name-pattern
391+
A regular expression that configures the test runner to only execute tests
392+
whose name matches the provided pattern.
393+
.
390394
.It Fl -test-only
391395
Configures the test runner to only execute top level tests that have the `only`
392396
option set.
Collapse file

‎lib/internal/test_runner/test.js‎

Copy file name to clipboardExpand all lines: lib/internal/test_runner/test.js
+28-3Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
'use strict';
22
const {
3+
ArrayPrototypeMap,
34
ArrayPrototypePush,
45
ArrayPrototypeReduce,
56
ArrayPrototypeShift,
67
ArrayPrototypeSlice,
8+
ArrayPrototypeSome,
79
ArrayPrototypeUnshift,
810
FunctionPrototype,
911
MathMax,
@@ -12,6 +14,7 @@ const {
1214
PromisePrototypeThen,
1315
PromiseResolve,
1416
ReflectApply,
17+
RegExpPrototypeExec,
1518
SafeMap,
1619
SafeSet,
1720
SafePromiseAll,
@@ -30,7 +33,11 @@ const {
3033
} = require('internal/errors');
3134
const { getOptionValue } = require('internal/options');
3235
const { TapStream } = require('internal/test_runner/tap_stream');
33-
const { createDeferredCallback, isTestFailureError } = require('internal/test_runner/utils');
36+
const {
37+
convertStringToRegExp,
38+
createDeferredCallback,
39+
isTestFailureError,
40+
} = require('internal/test_runner/utils');
3441
const {
3542
createDeferredPromise,
3643
kEmptyObject,
@@ -58,6 +65,13 @@ const kDefaultTimeout = null;
5865
const noop = FunctionPrototype;
5966
const isTestRunner = getOptionValue('--test');
6067
const testOnlyFlag = !isTestRunner && getOptionValue('--test-only');
68+
const testNamePatternFlag = isTestRunner ? null :
69+
getOptionValue('--test-name-pattern');
70+
const testNamePatterns = testNamePatternFlag?.length > 0 ?
71+
ArrayPrototypeMap(
72+
testNamePatternFlag,
73+
(re) => convertStringToRegExp(re, '--test-name-pattern')
74+
) : null;
6175
const kShouldAbort = Symbol('kShouldAbort');
6276
const kRunHook = Symbol('kRunHook');
6377
const kHookNames = ObjectSeal(['before', 'after', 'beforeEach', 'afterEach']);
@@ -195,6 +209,18 @@ class Test extends AsyncResource {
195209
this.timeout = timeout;
196210
}
197211

212+
if (testNamePatterns !== null) {
213+
// eslint-disable-next-line no-use-before-define
214+
const match = this instanceof TestHook || ArrayPrototypeSome(
215+
testNamePatterns,
216+
(re) => RegExpPrototypeExec(re, name) !== null
217+
);
218+
219+
if (!match) {
220+
skip = 'test name does not match pattern';
221+
}
222+
}
223+
198224
if (testOnlyFlag && !this.only) {
199225
skip = '\'only\' option not set';
200226
}
@@ -210,7 +236,6 @@ class Test extends AsyncResource {
210236
validateAbortSignal(signal, 'options.signal');
211237
this.#outerSignal?.addEventListener('abort', this.#abortHandler);
212238

213-
214239
this.fn = fn;
215240
this.name = name;
216241
this.parent = parent;
@@ -669,6 +694,7 @@ class ItTest extends Test {
669694
return { ctx: { signal: this.signal, name: this.name }, args: [] };
670695
}
671696
}
697+
672698
class Suite extends Test {
673699
constructor(options) {
674700
super(options);
@@ -704,7 +730,6 @@ class Suite extends Test {
704730
return;
705731
}
706732

707-
708733
const hookArgs = this.getRunArgs();
709734
await this[kRunHook]('before', hookArgs);
710735
const stopPromise = stopTest(this.timeout, this.signal);
Collapse file

‎lib/internal/test_runner/utils.js‎

Copy file name to clipboardExpand all lines: lib/internal/test_runner/utils.js
+22-1Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,17 @@
11
'use strict';
2-
const { RegExpPrototypeExec } = primordials;
2+
const { RegExp, RegExpPrototypeExec } = primordials;
33
const { basename } = require('path');
44
const { createDeferredPromise } = require('internal/util');
55
const {
66
codes: {
7+
ERR_INVALID_ARG_VALUE,
78
ERR_TEST_FAILURE,
89
},
910
kIsNodeError,
1011
} = require('internal/errors');
1112

1213
const kMultipleCallbackInvocations = 'multipleCallbackInvocations';
14+
const kRegExpPattern = /^\/(.*)\/([a-z]*)$/;
1315
const kSupportedFileExtensions = /\.[cm]?js$/;
1416
const kTestFilePattern = /((^test(-.+)?)|(.+[.\-_]test))\.[cm]?js$/;
1517

@@ -54,7 +56,26 @@ function isTestFailureError(err) {
5456
return err?.code === 'ERR_TEST_FAILURE' && kIsNodeError in err;
5557
}
5658

59+
function convertStringToRegExp(str, name) {
60+
const match = RegExpPrototypeExec(kRegExpPattern, str);
61+
const pattern = match?.[1] ?? str;
62+
const flags = match?.[2] || '';
63+
64+
try {
65+
return new RegExp(pattern, flags);
66+
} catch (err) {
67+
const msg = err?.message;
68+
69+
throw new ERR_INVALID_ARG_VALUE(
70+
name,
71+
str,
72+
`is an invalid regular expression.${msg ? ` ${msg}` : ''}`
73+
);
74+
}
75+
}
76+
5777
module.exports = {
78+
convertStringToRegExp,
5879
createDeferredCallback,
5980
doesPathMatchFilter,
6081
isSupportedFileType,
Collapse file

‎src/node_options.cc‎

Copy file name to clipboardExpand all lines: src/node_options.cc
+3Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -553,6 +553,9 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
553553
AddOption("--test",
554554
"launch test runner on startup",
555555
&EnvironmentOptions::test_runner);
556+
AddOption("--test-name-pattern",
557+
"run tests whose name matches this regular expression",
558+
&EnvironmentOptions::test_name_pattern);
556559
AddOption("--test-only",
557560
"run tests with 'only' option set",
558561
&EnvironmentOptions::test_only,
Collapse file

‎src/node_options.h‎

Copy file name to clipboardExpand all lines: src/node_options.h
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,7 @@ class EnvironmentOptions : public Options {
153153
std::string redirect_warnings;
154154
std::string diagnostic_dir;
155155
bool test_runner = false;
156+
std::vector<std::string> test_name_pattern;
156157
bool test_only = false;
157158
bool test_udp_no_try_send = false;
158159
bool throw_deprecation = false;
Collapse file

‎test/message/test_runner_test_name_pattern.js‎

Copy file name to clipboard
+47Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
// Flags: --no-warnings --test-name-pattern=enabled --test-name-pattern=/pattern/i
2+
'use strict';
3+
const common = require('../common');
4+
const {
5+
after,
6+
afterEach,
7+
before,
8+
beforeEach,
9+
describe,
10+
it,
11+
test,
12+
} = require('node:test');
13+
14+
test('top level test disabled', common.mustNotCall());
15+
test('top level skipped test disabled', { skip: true }, common.mustNotCall());
16+
test('top level skipped test enabled', { skip: true }, common.mustNotCall());
17+
it('top level it enabled', common.mustCall());
18+
it('top level it disabled', common.mustNotCall());
19+
it.skip('top level skipped it disabled', common.mustNotCall());
20+
it.skip('top level skipped it enabled', common.mustNotCall());
21+
describe('top level describe disabled', common.mustNotCall());
22+
describe.skip('top level skipped describe disabled', common.mustNotCall());
23+
describe.skip('top level skipped describe enabled', common.mustNotCall());
24+
test('top level runs because name includes PaTtErN', common.mustCall());
25+
26+
test('top level test enabled', common.mustCall(async (t) => {
27+
t.beforeEach(common.mustCall());
28+
t.afterEach(common.mustCall());
29+
await t.test(
30+
'nested test runs because name includes PATTERN',
31+
common.mustCall()
32+
);
33+
}));
34+
35+
describe('top level describe enabled', () => {
36+
before(common.mustCall());
37+
beforeEach(common.mustCall(2));
38+
afterEach(common.mustCall(2));
39+
after(common.mustCall());
40+
41+
it('nested it disabled', common.mustNotCall());
42+
it('nested it enabled', common.mustCall());
43+
describe('nested describe disabled', common.mustNotCall());
44+
describe('nested describe enabled', common.mustCall(() => {
45+
it('is enabled', common.mustCall());
46+
}));
47+
});
Collapse file

‎test/message/test_runner_test_name_pattern.out‎

Copy file name to clipboard
+107Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
TAP version 13
2+
# Subtest: top level test disabled
3+
ok 1 - top level test disabled # SKIP test name does not match pattern
4+
---
5+
duration_ms: *
6+
...
7+
# Subtest: top level skipped test disabled
8+
ok 2 - top level skipped test disabled # SKIP test name does not match pattern
9+
---
10+
duration_ms: *
11+
...
12+
# Subtest: top level skipped test enabled
13+
ok 3 - top level skipped test enabled # SKIP
14+
---
15+
duration_ms: *
16+
...
17+
# Subtest: top level it enabled
18+
ok 4 - top level it enabled
19+
---
20+
duration_ms: *
21+
...
22+
# Subtest: top level it disabled
23+
ok 5 - top level it disabled # SKIP test name does not match pattern
24+
---
25+
duration_ms: *
26+
...
27+
# Subtest: top level skipped it disabled
28+
ok 6 - top level skipped it disabled # SKIP test name does not match pattern
29+
---
30+
duration_ms: *
31+
...
32+
# Subtest: top level skipped it enabled
33+
ok 7 - top level skipped it enabled # SKIP
34+
---
35+
duration_ms: *
36+
...
37+
# Subtest: top level describe disabled
38+
ok 8 - top level describe disabled # SKIP test name does not match pattern
39+
---
40+
duration_ms: *
41+
...
42+
# Subtest: top level skipped describe disabled
43+
ok 9 - top level skipped describe disabled # SKIP test name does not match pattern
44+
---
45+
duration_ms: *
46+
...
47+
# Subtest: top level skipped describe enabled
48+
ok 10 - top level skipped describe enabled # SKIP
49+
---
50+
duration_ms: *
51+
...
52+
# Subtest: top level runs because name includes PaTtErN
53+
ok 11 - top level runs because name includes PaTtErN
54+
---
55+
duration_ms: *
56+
...
57+
# Subtest: top level test enabled
58+
# Subtest: nested test runs because name includes PATTERN
59+
ok 1 - nested test runs because name includes PATTERN
60+
---
61+
duration_ms: *
62+
...
63+
1..1
64+
ok 12 - top level test enabled
65+
---
66+
duration_ms: *
67+
...
68+
# Subtest: top level describe enabled
69+
# Subtest: nested it disabled
70+
ok 1 - nested it disabled # SKIP test name does not match pattern
71+
---
72+
duration_ms: *
73+
...
74+
# Subtest: nested it enabled
75+
ok 2 - nested it enabled
76+
---
77+
duration_ms: *
78+
...
79+
# Subtest: nested describe disabled
80+
ok 3 - nested describe disabled # SKIP test name does not match pattern
81+
---
82+
duration_ms: *
83+
...
84+
# Subtest: nested describe enabled
85+
# Subtest: is enabled
86+
ok 1 - is enabled
87+
---
88+
duration_ms: *
89+
...
90+
1..1
91+
ok 4 - nested describe enabled
92+
---
93+
duration_ms: *
94+
...
95+
1..4
96+
ok 13 - top level describe enabled
97+
---
98+
duration_ms: *
99+
...
100+
1..13
101+
# tests 13
102+
# pass 4
103+
# fail 0
104+
# cancelled 0
105+
# skipped 9
106+
# todo 0
107+
# duration_ms *

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.