Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 9bc2cec

Browse filesBrowse files
tarrudadanielleadams
tarruda
authored and
danielleadams
committed
child_process: add 'overlapped' stdio flag
The 'overlapped' value sets the UV_OVERLAPPED_PIPE libuv flag in the child process stdio. Fixes: #29238 PR-URL: #29412 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
1 parent 8b43388 commit 9bc2cec
Copy full SHA for 9bc2cec

File tree

Expand file treeCollapse file tree

8 files changed

+255
-8
lines changed
Open diff view settings
Filter options
Expand file treeCollapse file tree

8 files changed

+255
-8
lines changed
Open diff view settings
Collapse file

‎doc/api/child_process.md‎

Copy file name to clipboardExpand all lines: doc/api/child_process.md
+16-6Lines changed: 16 additions & 6 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -660,6 +660,9 @@ subprocess.unref();
660660
<!-- YAML
661661
added: v0.7.10
662662
changes:
663+
- version: REPLACEME
664+
pr-url: https://github.com/nodejs/node/pull/29412
665+
description: Added the `overlapped` stdio flag.
663666
- version: v3.3.1
664667
pr-url: https://github.com/nodejs/node/pull/2727
665668
description: The value `0` is now accepted as a file descriptor.
@@ -675,6 +678,7 @@ equal to `['pipe', 'pipe', 'pipe']`.
675678
For convenience, `options.stdio` may be one of the following strings:
676679

677680
* `'pipe'`: equivalent to `['pipe', 'pipe', 'pipe']` (the default)
681+
* `'overlapped'`: equivalent to `['overlapped', 'overlapped', 'overlapped']`
678682
* `'ignore'`: equivalent to `['ignore', 'ignore', 'ignore']`
679683
* `'inherit'`: equivalent to `['inherit', 'inherit', 'inherit']` or `[0, 1, 2]`
680684

@@ -688,7 +692,13 @@ pipes between the parent and child. The value is one of the following:
688692
`child_process` object as [`subprocess.stdio[fd]`][`subprocess.stdio`]. Pipes
689693
created for fds 0, 1, and 2 are also available as [`subprocess.stdin`][],
690694
[`subprocess.stdout`][] and [`subprocess.stderr`][], respectively.
691-
2. `'ipc'`: Create an IPC channel for passing messages/file descriptors
695+
1. `'overlapped'`: Same as `'pipe'` except that the `FILE_FLAG_OVERLAPPED` flag
696+
is set on the handle. This is necessary for overlapped I/O on the child
697+
process's stdio handles. See the
698+
[docs](https://docs.microsoft.com/en-us/windows/win32/fileio/synchronous-and-asynchronous-i-o)
699+
for more details. This is exactly the same as `'pipe'` on non-Windows
700+
systems.
701+
1. `'ipc'`: Create an IPC channel for passing messages/file descriptors
692702
between parent and child. A [`ChildProcess`][] may have at most one IPC
693703
stdio file descriptor. Setting this option enables the
694704
[`subprocess.send()`][] method. If the child is a Node.js process, the
@@ -699,25 +709,25 @@ pipes between the parent and child. The value is one of the following:
699709
Accessing the IPC channel fd in any way other than [`process.send()`][]
700710
or using the IPC channel with a child process that is not a Node.js instance
701711
is not supported.
702-
3. `'ignore'`: Instructs Node.js to ignore the fd in the child. While Node.js
712+
1. `'ignore'`: Instructs Node.js to ignore the fd in the child. While Node.js
703713
will always open fds 0, 1, and 2 for the processes it spawns, setting the fd
704714
to `'ignore'` will cause Node.js to open `/dev/null` and attach it to the
705715
child's fd.
706-
4. `'inherit'`: Pass through the corresponding stdio stream to/from the
716+
1. `'inherit'`: Pass through the corresponding stdio stream to/from the
707717
parent process. In the first three positions, this is equivalent to
708718
`process.stdin`, `process.stdout`, and `process.stderr`, respectively. In
709719
any other position, equivalent to `'ignore'`.
710-
5. {Stream} object: Share a readable or writable stream that refers to a tty,
720+
1. {Stream} object: Share a readable or writable stream that refers to a tty,
711721
file, socket, or a pipe with the child process. The stream's underlying
712722
file descriptor is duplicated in the child process to the fd that
713723
corresponds to the index in the `stdio` array. The stream must have an
714724
underlying descriptor (file streams do not until the `'open'` event has
715725
occurred).
716-
6. Positive integer: The integer value is interpreted as a file descriptor
726+
1. Positive integer: The integer value is interpreted as a file descriptor
717727
that is currently open in the parent process. It is shared with the child
718728
process, similar to how {Stream} objects can be shared. Passing sockets
719729
is not supported on Windows.
720-
7. `null`, `undefined`: Use default value. For stdio fds 0, 1, and 2 (in other
730+
1. `null`, `undefined`: Use default value. For stdio fds 0, 1, and 2 (in other
721731
words, stdin, stdout, and stderr) a pipe is created. For fd 3 and up, the
722732
default is `'ignore'`.
723733

Collapse file

‎lib/internal/child_process.js‎

Copy file name to clipboardExpand all lines: lib/internal/child_process.js
+4-2Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,7 @@ function stdioStringToArray(stdio, channel) {
231231

232232
switch (stdio) {
233233
case 'ignore':
234+
case 'overlapped':
234235
case 'pipe': ArrayPrototypePush(options, stdio, stdio, stdio); break;
235236
case 'inherit': ArrayPrototypePush(options, 0, 1, 2); break;
236237
default:
@@ -976,9 +977,10 @@ function getValidStdio(stdio, sync) {
976977

977978
if (stdio === 'ignore') {
978979
ArrayPrototypePush(acc, { type: 'ignore' });
979-
} else if (stdio === 'pipe' || (typeof stdio === 'number' && stdio < 0)) {
980+
} else if (stdio === 'pipe' || stdio === 'overlapped' ||
981+
(typeof stdio === 'number' && stdio < 0)) {
980982
const a = {
981-
type: 'pipe',
983+
type: stdio === 'overlapped' ? 'overlapped' : 'pipe',
982984
readable: i === 0,
983985
writable: i !== 0
984986
};
Collapse file

‎node.gyp‎

Copy file name to clipboardExpand all lines: node.gyp
+18Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1469,6 +1469,24 @@
14691469
],
14701470
}, # embedtest
14711471

1472+
{
1473+
'target_name': 'overlapped-checker',
1474+
'type': 'executable',
1475+
1476+
'conditions': [
1477+
['OS=="win"', {
1478+
'sources': [
1479+
'test/overlapped-checker/main_win.c'
1480+
],
1481+
}],
1482+
['OS!="win"', {
1483+
'sources': [
1484+
'test/overlapped-checker/main_unix.c'
1485+
],
1486+
}],
1487+
]
1488+
}, # overlapped-checker
1489+
14721490
# TODO(joyeecheung): do not depend on node_lib,
14731491
# instead create a smaller static library node_lib_base that does
14741492
# just enough for node_native_module.cc and the cache builder to
Collapse file

‎src/env.h‎

Copy file name to clipboardExpand all lines: src/env.h
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -343,6 +343,7 @@ constexpr size_t kFsStatsBufferLength =
343343
V(options_string, "options") \
344344
V(order_string, "order") \
345345
V(output_string, "output") \
346+
V(overlapped_string, "overlapped") \
346347
V(parse_error_string, "Parse Error") \
347348
V(password_string, "password") \
348349
V(path_string, "path") \
Collapse file

‎src/process_wrap.cc‎

Copy file name to clipboardExpand all lines: src/process_wrap.cc
+5Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,11 @@ class ProcessWrap : public HandleWrap {
120120
options->stdio[i].flags = static_cast<uv_stdio_flags>(
121121
UV_CREATE_PIPE | UV_READABLE_PIPE | UV_WRITABLE_PIPE);
122122
options->stdio[i].data.stream = StreamForWrap(env, stdio);
123+
} else if (type->StrictEquals(env->overlapped_string())) {
124+
options->stdio[i].flags = static_cast<uv_stdio_flags>(
125+
UV_CREATE_PIPE | UV_READABLE_PIPE | UV_WRITABLE_PIPE |
126+
UV_OVERLAPPED_PIPE);
127+
options->stdio[i].data.stream = StreamForWrap(env, stdio);
123128
} else if (type->StrictEquals(env->wrap_string())) {
124129
options->stdio[i].flags = UV_INHERIT_STREAM;
125130
options->stdio[i].data.stream = StreamForWrap(env, stdio);
Collapse file

‎test/overlapped-checker/main_unix.c‎

Copy file name to clipboard
+51Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#include <stdio.h>
2+
#include <string.h>
3+
#include <stdlib.h>
4+
5+
#include <errno.h>
6+
#include <unistd.h>
7+
8+
static size_t r(char* buf, size_t buf_size) {
9+
ssize_t read_count;
10+
do
11+
read_count = read(0, buf, buf_size);
12+
while (read_count < 0 && errno == EINTR);
13+
if (read_count <= 0)
14+
abort();
15+
return (size_t)read_count;
16+
}
17+
18+
static void w(const char* buf, size_t count) {
19+
const char* end = buf + count;
20+
21+
while (buf < end) {
22+
ssize_t write_count;
23+
do
24+
write_count = write(1, buf, count);
25+
while (write_count < 0 && errno == EINTR);
26+
if (write_count <= 0)
27+
abort();
28+
buf += write_count;
29+
}
30+
31+
fprintf(stderr, "%zu", count);
32+
fflush(stderr);
33+
}
34+
35+
int main(void) {
36+
w("0", 1);
37+
38+
while (1) {
39+
char buf[256];
40+
size_t read_count = r(buf, sizeof(buf));
41+
// The JS part (test-child-process-stdio-overlapped.js) only writes the
42+
// "exit" string when the buffer is empty, so the read is guaranteed to be
43+
// atomic due to it being less than PIPE_BUF.
44+
if (!strncmp(buf, "exit", read_count)) {
45+
break;
46+
}
47+
w(buf, read_count);
48+
}
49+
50+
return 0;
51+
}
Collapse file

‎test/overlapped-checker/main_win.c‎

Copy file name to clipboard
+85Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
#include <stdlib.h>
2+
#include <stdio.h>
3+
#include <string.h>
4+
5+
#include <windows.h>
6+
7+
static char buf[256];
8+
static DWORD read_count;
9+
static DWORD write_count;
10+
static HANDLE stdin_h;
11+
static OVERLAPPED stdin_o;
12+
13+
static void die(const char* buf) {
14+
fprintf(stderr, "%s\n", buf);
15+
fflush(stderr);
16+
exit(100);
17+
}
18+
19+
static void overlapped_read(void) {
20+
if (ReadFile(stdin_h, buf, sizeof(buf), NULL, &stdin_o)) {
21+
// Since we start the read operation immediately before requesting a write,
22+
// it should never complete synchronously since no data would be available
23+
die("read completed synchronously");
24+
}
25+
if (GetLastError() != ERROR_IO_PENDING) {
26+
die("overlapped read failed");
27+
}
28+
}
29+
30+
static void write(const char* buf, size_t buf_size) {
31+
overlapped_read();
32+
DWORD write_count;
33+
HANDLE stdout_h = GetStdHandle(STD_OUTPUT_HANDLE);
34+
if (!WriteFile(stdout_h, buf, buf_size, &write_count, NULL)) {
35+
die("overlapped write failed");
36+
}
37+
fprintf(stderr, "%d", write_count);
38+
fflush(stderr);
39+
}
40+
41+
int main(void) {
42+
HANDLE event = CreateEvent(NULL, FALSE, FALSE, NULL);
43+
if (event == NULL) {
44+
die("failed to create event handle");
45+
}
46+
47+
stdin_h = GetStdHandle(STD_INPUT_HANDLE);
48+
stdin_o.hEvent = event;
49+
50+
write("0", 1);
51+
52+
while (1) {
53+
DWORD result = WaitForSingleObject(event, INFINITE);
54+
if (result == WAIT_OBJECT_0) {
55+
if (!GetOverlappedResult(stdin_h, &stdin_o, &read_count, FALSE)) {
56+
die("failed to get overlapped read result");
57+
}
58+
if (strncmp(buf, "exit", read_count) == 0) {
59+
break;
60+
}
61+
write(buf, read_count);
62+
} else {
63+
char emsg[0xfff];
64+
int ecode = GetLastError();
65+
DWORD rv = FormatMessage(
66+
FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
67+
NULL,
68+
ecode,
69+
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
70+
(LPSTR)emsg,
71+
sizeof(emsg),
72+
NULL);
73+
if (rv > 0) {
74+
snprintf(emsg, sizeof(emsg),
75+
"WaitForSingleObject failed. Error %d (%s)", ecode, emsg);
76+
} else {
77+
snprintf(emsg, sizeof(emsg),
78+
"WaitForSingleObject failed. Error %d", ecode);
79+
}
80+
die(emsg);
81+
}
82+
}
83+
84+
return 0;
85+
}
Collapse file

‎test/parallel/test-child-process-stdio-overlapped.js‎

Copy file name to clipboard
+75Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
// Test for "overlapped" stdio option. This test uses the "overlapped-checker"
2+
// helper program which basically a specialized echo program.
3+
//
4+
// The test has two goals:
5+
//
6+
// - Verify that overlapped I/O works on windows. The test program will deadlock
7+
// if stdin doesn't have the FILE_FLAG_OVERLAPPED flag set on startup (see
8+
// test/overlapped-checker/main_win.c for more details).
9+
// - Verify that "overlapped" stdio option works transparently as a pipe (on
10+
// unix/windows)
11+
//
12+
// This is how the test works:
13+
//
14+
// - This script assumes only numeric strings are written to the test program
15+
// stdout.
16+
// - The test program will be spawned with "overlapped" set on stdin and "pipe"
17+
// set on stdout/stderr and at startup writes a number to its stdout
18+
// - When this script receives some data, it will parse the number, add 50 and
19+
// write to the test program's stdin.
20+
// - The test program will then echo the number back to us which will repeat the
21+
// cycle until the number reaches 200, at which point we send the "exit"
22+
// string, which causes the test program to exit.
23+
// - Extra assertion: Every time the test program writes a string to its stdout,
24+
// it will write the number of bytes written to stderr.
25+
// - If overlapped I/O is not setup correctly, this test is going to hang.
26+
'use strict';
27+
const common = require('../common');
28+
const assert = require('assert');
29+
const path = require('path');
30+
const child_process = require('child_process');
31+
32+
const exeExtension = process.platform === 'win32' ? '.exe' : '';
33+
const exe = 'overlapped-checker' + exeExtension;
34+
const exePath = path.join(path.dirname(process.execPath), exe);
35+
36+
const child = child_process.spawn(exePath, [], {
37+
stdio: ['overlapped', 'pipe', 'pipe']
38+
});
39+
40+
child.stdin.setEncoding('utf8');
41+
child.stdout.setEncoding('utf8');
42+
child.stderr.setEncoding('utf8');
43+
44+
function writeNext(n) {
45+
child.stdin.write((n + 50).toString());
46+
}
47+
48+
child.stdout.on('data', (s) => {
49+
const n = Number(s);
50+
if (n >= 200) {
51+
child.stdin.write('exit');
52+
return;
53+
}
54+
writeNext(n);
55+
});
56+
57+
let stderr = '';
58+
child.stderr.on('data', (s) => {
59+
stderr += s;
60+
});
61+
62+
child.stderr.on('end', common.mustCall(() => {
63+
// This is the sequence of numbers sent to us:
64+
// - 0 (1 byte written)
65+
// - 50 (2 bytes written)
66+
// - 100 (3 bytes written)
67+
// - 150 (3 bytes written)
68+
// - 200 (3 bytes written)
69+
assert.strictEqual(stderr, '12333');
70+
}));
71+
72+
child.on('exit', common.mustCall((status) => {
73+
// The test program will return the number of writes as status code.
74+
assert.strictEqual(status, 0);
75+
}));

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.