Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 886ef09

Browse filesBrowse files
addaleaxMylesBorins
addaleax
authored and
MylesBorins
committed
worker: allow specifying resource limits
Allow specifying resource limits for the JS engine instance created as part of a Worker. PR-URL: #26628 Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
1 parent fcc4bf9 commit 886ef09
Copy full SHA for 886ef09

File tree

Expand file treeCollapse file tree

8 files changed

+315
-26
lines changed
Open diff view settings
Filter options
Expand file treeCollapse file tree

8 files changed

+315
-26
lines changed
Open diff view settings
Collapse file

‎doc/api/errors.md‎

Copy file name to clipboardExpand all lines: doc/api/errors.md
+5Lines changed: 5 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -1990,6 +1990,11 @@ meaning of the error depends on the specific function.
19901990
The `execArgv` option passed to the `Worker` constructor contains
19911991
invalid flags.
19921992

1993+
<a id="ERR_WORKER_OUT_OF_MEMORY"></a>
1994+
### `ERR_WORKER_OUT_OF_MEMORY`
1995+
1996+
The `Worker` instance terminated because it reached its memory limit.
1997+
19931998
<a id="ERR_WORKER_PATH"></a>
19941999
### `ERR_WORKER_PATH`
19952000

Collapse file

‎doc/api/worker_threads.md‎

Copy file name to clipboardExpand all lines: doc/api/worker_threads.md
+48Lines changed: 48 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,22 @@ console.log(receiveMessageOnPort(port2));
157157
When this function is used, no `'message'` event will be emitted and the
158158
`onmessage` listener will not be invoked.
159159

160+
### `worker.resourceLimits`
161+
<!-- YAML
162+
added: REPLACEME
163+
-->
164+
165+
* {Object|undefined}
166+
* `maxYoungGenerationSizeMb` {number}
167+
* `maxOldGenerationSizeMb` {number}
168+
* `codeRangeSizeMb` {number}
169+
170+
Provides the set of JS engine resource constraints inside this Worker thread.
171+
If the `resourceLimits` option was passed to the [`Worker`][] constructor,
172+
this matches its values.
173+
174+
If this is used in the main thread, its value is an empty object.
175+
160176
## `worker.SHARE_ENV`
161177
<!-- YAML
162178
added: v11.14.0
@@ -488,6 +504,13 @@ if (isMainThread) {
488504
```
489505

490506
### `new Worker(filename[, options])`
507+
<!-- YAML
508+
added: v10.5.0
509+
changes:
510+
- version: REPLACEME
511+
pr-url: https://github.com/nodejs/node/pull/26628
512+
description: The `resourceLimits` option was introduced.
513+
-->
491514

492515
* `filename` {string} The path to the Worker’s main script. Must be
493516
either an absolute path or a relative path (i.e. relative to the
@@ -519,6 +542,16 @@ if (isMainThread) {
519542
occur as described in the [HTML structured clone algorithm][], and an error
520543
will be thrown if the object cannot be cloned (e.g. because it contains
521544
`function`s).
545+
* `resourceLimits` {Object} An optional set of resource limits for the new
546+
JS engine instance. Reaching these limits will lead to termination of the
547+
`Worker` instance. These limits only affect the JS engine, and no external
548+
data, including no `ArrayBuffer`s. Even if these limits are set, the process
549+
may still abort if it encounters a global out-of-memory situation.
550+
* `maxOldGenerationSizeMb` {number} The maximum size of the main heap in MB.
551+
* `maxYoungGenerationSizeMb` {number} The maximum size of a heap space for
552+
recently created objects.
553+
* `codeRangeSizeMb` {number} The size of a pre-allocated memory range
554+
used for generated code.
522555

523556
### Event: `'error'`
524557
<!-- YAML
@@ -583,6 +616,21 @@ Opposite of `unref()`, calling `ref()` on a previously `unref()`ed worker will
583616
behavior). If the worker is `ref()`ed, calling `ref()` again will have
584617
no effect.
585618

619+
### `worker.resourceLimits`
620+
<!-- YAML
621+
added: REPLACEME
622+
-->
623+
624+
* {Object}
625+
* `maxYoungGenerationSizeMb` {number}
626+
* `maxOldGenerationSizeMb` {number}
627+
* `codeRangeSizeMb` {number}
628+
629+
Provides the set of JS engine resource constraints for this Worker thread.
630+
If the `resourceLimits` option was passed to the [`Worker`][] constructor,
631+
this matches its values.
632+
633+
If the worker has stopped, the return value is an empty object.
586634
### `worker.stderr`
587635
<!-- YAML
588636
added: v10.5.0
Collapse file

‎lib/internal/errors.js‎

Copy file name to clipboardExpand all lines: lib/internal/errors.js
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1218,6 +1218,8 @@ E('ERR_VM_MODULE_STATUS', 'Module status %s', Error);
12181218
E('ERR_WORKER_INVALID_EXEC_ARGV', (errors) =>
12191219
`Initiated Worker with invalid execArgv flags: ${errors.join(', ')}`,
12201220
Error);
1221+
E('ERR_WORKER_OUT_OF_MEMORY', 'Worker terminated due to reaching memory limit',
1222+
Error);
12211223
E('ERR_WORKER_PATH',
12221224
'The worker script filename must be an absolute path or a relative ' +
12231225
'path starting with \'./\' or \'../\'. Received "%s"',
Collapse file

‎lib/internal/worker.js‎

Copy file name to clipboardExpand all lines: lib/internal/worker.js
+47-5Lines changed: 47 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,20 @@
22

33
/* global SharedArrayBuffer */
44

5-
const { Object } = primordials;
5+
const { Math, Object } = primordials;
66

77
const EventEmitter = require('events');
88
const assert = require('internal/assert');
99
const path = require('path');
1010

11+
const errorCodes = require('internal/errors').codes;
1112
const {
1213
ERR_WORKER_PATH,
1314
ERR_WORKER_UNSERIALIZABLE_ERROR,
1415
ERR_WORKER_UNSUPPORTED_EXTENSION,
1516
ERR_WORKER_INVALID_EXEC_ARGV,
1617
ERR_INVALID_ARG_TYPE,
17-
} = require('internal/errors').codes;
18+
} = errorCodes;
1819
const { validateString } = require('internal/validators');
1920
const { getOptionValue } = require('internal/options');
2021

@@ -37,8 +38,13 @@ const { pathToFileURL } = require('url');
3738
const {
3839
ownsProcessState,
3940
isMainThread,
41+
resourceLimits: resourceLimitsRaw,
4042
threadId,
4143
Worker: WorkerImpl,
44+
kMaxYoungGenerationSizeMb,
45+
kMaxOldGenerationSizeMb,
46+
kCodeRangeSizeMb,
47+
kTotalResourceLimitCount
4248
} = internalBinding('worker');
4349

4450
const kHandle = Symbol('kHandle');
@@ -102,7 +108,8 @@ class Worker extends EventEmitter {
102108

103109
const url = options.eval ? null : pathToFileURL(filename);
104110
// Set up the C++ handle for the worker, as well as some internal wiring.
105-
this[kHandle] = new WorkerImpl(url, options.execArgv);
111+
this[kHandle] = new WorkerImpl(url, options.execArgv,
112+
parseResourceLimits(options.resourceLimits));
106113
if (this[kHandle].invalidExecArgv) {
107114
throw new ERR_WORKER_INVALID_EXEC_ARGV(this[kHandle].invalidExecArgv);
108115
}
@@ -113,7 +120,7 @@ class Worker extends EventEmitter {
113120
} else if (env !== undefined) {
114121
this[kHandle].setEnvVars(env);
115122
}
116-
this[kHandle].onexit = (code) => this[kOnExit](code);
123+
this[kHandle].onexit = (code, customErr) => this[kOnExit](code, customErr);
117124
this[kPort] = this[kHandle].messagePort;
118125
this[kPort].on('message', (data) => this[kOnMessage](data));
119126
this[kPort].start();
@@ -157,11 +164,15 @@ class Worker extends EventEmitter {
157164
this[kHandle].startThread();
158165
}
159166

160-
[kOnExit](code) {
167+
[kOnExit](code, customErr) {
161168
debug(`[${threadId}] hears end event for Worker ${this.threadId}`);
162169
drainMessagePort(this[kPublicPort]);
163170
drainMessagePort(this[kPort]);
164171
this[kDispose]();
172+
if (customErr) {
173+
debug(`[${threadId}] failing with custom error ${customErr}`);
174+
this.emit('error', new errorCodes[customErr]());
175+
}
165176
this.emit('exit', code);
166177
this.removeAllListeners();
167178
}
@@ -280,6 +291,12 @@ class Worker extends EventEmitter {
280291
get stderr() {
281292
return this[kParentSideStdio].stderr;
282293
}
294+
295+
get resourceLimits() {
296+
if (this[kHandle] === null) return {};
297+
298+
return makeResourceLimits(this[kHandle].getResourceLimits());
299+
}
283300
}
284301

285302
function pipeWithoutWarning(source, dest) {
@@ -294,10 +311,35 @@ function pipeWithoutWarning(source, dest) {
294311
dest._maxListeners = destMaxListeners;
295312
}
296313

314+
const resourceLimitsArray = new Float64Array(kTotalResourceLimitCount);
315+
function parseResourceLimits(obj) {
316+
const ret = resourceLimitsArray;
317+
ret.fill(-1);
318+
if (typeof obj !== 'object' || obj === null) return ret;
319+
320+
if (typeof obj.maxOldGenerationSizeMb === 'number')
321+
ret[kMaxOldGenerationSizeMb] = Math.max(obj.maxOldGenerationSizeMb, 2);
322+
if (typeof obj.maxYoungGenerationSizeMb === 'number')
323+
ret[kMaxYoungGenerationSizeMb] = obj.maxYoungGenerationSizeMb;
324+
if (typeof obj.codeRangeSizeMb === 'number')
325+
ret[kCodeRangeSizeMb] = obj.codeRangeSizeMb;
326+
return ret;
327+
}
328+
329+
function makeResourceLimits(float64arr) {
330+
return {
331+
maxYoungGenerationSizeMb: float64arr[kMaxYoungGenerationSizeMb],
332+
maxOldGenerationSizeMb: float64arr[kMaxOldGenerationSizeMb],
333+
codeRangeSizeMb: float64arr[kCodeRangeSizeMb]
334+
};
335+
}
336+
297337
module.exports = {
298338
ownsProcessState,
299339
isMainThread,
300340
SHARE_ENV,
341+
resourceLimits:
342+
!isMainThread ? makeResourceLimits(resourceLimitsRaw) : {},
301343
threadId,
302344
Worker,
303345
};
Collapse file

‎lib/worker_threads.js‎

Copy file name to clipboardExpand all lines: lib/worker_threads.js
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
const {
44
isMainThread,
55
SHARE_ENV,
6+
resourceLimits,
67
threadId,
78
Worker
89
} = require('internal/worker');
@@ -20,6 +21,7 @@ module.exports = {
2021
MessageChannel,
2122
moveMessagePortToContext,
2223
receiveMessageOnPort,
24+
resourceLimits,
2325
threadId,
2426
SHARE_ENV,
2527
Worker,

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.