Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit a7c8322

Browse filesBrowse files
bmeckBridgeAR
bmeck
authored and
BridgeAR
committed
esm: support loading data URLs
Co-Authored-By: Jan Olaf Krems <jan.krems@gmail.com> PR-URL: #28614 Reviewed-By: Jan Krems <jan.krems@gmail.com>
1 parent 6ff803d commit a7c8322
Copy full SHA for a7c8322

File tree

Expand file treeCollapse file tree

5 files changed

+179
-29
lines changed
Open diff view settings
Filter options
Expand file treeCollapse file tree

5 files changed

+179
-29
lines changed
Open diff view settings
Collapse file

‎doc/api/esm.md‎

Copy file name to clipboardExpand all lines: doc/api/esm.md
+29-1Lines changed: 29 additions & 1 deletion
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -312,13 +312,38 @@ There are four types of specifiers:
312312
Bare specifiers, and the bare specifier portion of deep import specifiers, are
313313
strings; but everything else in a specifier is a URL.
314314

315-
Only `file://` URLs are supported. A specifier like
315+
Only `file:` and `data:` URLs are supported. A specifier like
316316
`'https://example.com/app.js'` may be supported by browsers but it is not
317317
supported in Node.js.
318318

319319
Specifiers may not begin with `/` or `//`. These are reserved for potential
320320
future use. The root of the current volume may be referenced via `file:///`.
321321

322+
#### `data:` Imports
323+
324+
<!-- YAML
325+
added: REPLACEME
326+
-->
327+
328+
[`data:` URLs][] are supported for importing with the following MIME types:
329+
330+
* `text/javascript` for ES Modules
331+
* `application/json` for JSON
332+
* `application/wasm` for WASM.
333+
334+
`data:` URLs only resolve [_Bare specifiers_][Terminology] for builtin modules
335+
and [_Absolute specifiers_][Terminology]. Resolving
336+
[_Relative specifiers_][Terminology] will not work because `data:` is not a
337+
[special scheme][]. For example, attempting to load `./foo`
338+
from `data:text/javascript,import "./foo";` will fail to resolve since there
339+
is no concept of relative resolution for `data:` URLs. An example of a `data:`
340+
URLs being used is:
341+
342+
```mjs
343+
import 'data:text/javascript,console.log("hello!");'
344+
import _ from 'data:application/json,"world!"'
345+
```
346+
322347
## import.meta
323348

324349
* {Object}
@@ -869,6 +894,8 @@ $ node --experimental-modules --es-module-specifier-resolution=node index
869894
success!
870895
```
871896
897+
[Terminology]: #esm_terminology
898+
[`data:` URLs]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs
872899
[`export`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export
873900
[`import`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import
874901
[`import()`]: #esm_import-expressions
@@ -877,6 +904,7 @@ success!
877904
[CommonJS]: modules.html
878905
[ECMAScript-modules implementation]: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md
879906
[Node.js EP for ES Modules]: https://github.com/nodejs/node-eps/blob/master/002-es-modules.md
907+
[special scheme]: https://url.spec.whatwg.org/#special-scheme
880908
[WHATWG JSON modules specification]: https://html.spec.whatwg.org/#creating-a-json-module-script
881909
[ES Module Integration Proposal for Web Assembly]: https://github.com/webassembly/esm-integration
882910
[dynamic instantiate hook]: #esm_dynamic_instantiate_hook
Collapse file

‎lib/internal/modules/esm/default_resolve.js‎

Copy file name to clipboardExpand all lines: lib/internal/modules/esm/default_resolve.js
+21-1Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ const typeFlag = getOptionValue('--input-type');
1212
const experimentalWasmModules = getOptionValue('--experimental-wasm-modules');
1313
const { resolve: moduleWrapResolve,
1414
getPackageType } = internalBinding('module_wrap');
15-
const { pathToFileURL, fileURLToPath } = require('internal/url');
15+
const { URL, pathToFileURL, fileURLToPath } = require('internal/url');
1616
const { ERR_INPUT_TYPE_NOT_ALLOWED,
1717
ERR_UNKNOWN_FILE_EXTENSION } = require('internal/errors').codes;
1818

@@ -45,12 +45,32 @@ if (experimentalWasmModules)
4545
extensionFormatMap['.wasm'] = legacyExtensionFormatMap['.wasm'] = 'wasm';
4646

4747
function resolve(specifier, parentURL) {
48+
try {
49+
const parsed = new URL(specifier);
50+
if (parsed.protocol === 'data:') {
51+
const [ , mime ] = /^([^/]+\/[^;,]+)(;base64)?,/.exec(parsed.pathname) || [ null, null, null ];
52+
const format = ({
53+
'__proto__': null,
54+
'text/javascript': 'module',
55+
'application/json': 'json',
56+
'application/wasm': experimentalWasmModules ? 'wasm' : null
57+
})[mime] || null;
58+
return {
59+
url: specifier,
60+
format
61+
};
62+
}
63+
} catch {}
4864
if (NativeModule.canBeRequiredByUsers(specifier)) {
4965
return {
5066
url: specifier,
5167
format: 'builtin'
5268
};
5369
}
70+
if (parentURL && parentURL.startsWith('data:')) {
71+
// This is gonna blow up, we want the error
72+
new URL(specifier, parentURL);
73+
}
5474

5575
const isMain = parentURL === undefined;
5676
if (isMain)
Collapse file

‎lib/internal/modules/esm/loader.js‎

Copy file name to clipboardExpand all lines: lib/internal/modules/esm/loader.js
+5-2Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -102,9 +102,12 @@ class Loader {
102102
}
103103
}
104104

105-
if (format !== 'dynamic' && !url.startsWith('file:'))
105+
if (format !== 'dynamic' &&
106+
!url.startsWith('file:') &&
107+
!url.startsWith('data:')
108+
)
106109
throw new ERR_INVALID_RETURN_PROPERTY(
107-
'file: url', 'loader resolve', 'url', url
110+
'file: or data: url', 'loader resolve', 'url', url
108111
);
109112

110113
return { url, format };
Collapse file

‎lib/internal/modules/esm/translators.js‎

Copy file name to clipboardExpand all lines: lib/internal/modules/esm/translators.js
+61-25Lines changed: 61 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ const {
99
StringPrototype
1010
} = primordials;
1111

12+
const { Buffer } = require('buffer');
13+
1214
const {
1315
stripShebang,
1416
stripBOM,
@@ -24,6 +26,8 @@ const { debuglog } = require('internal/util/debuglog');
2426
const { promisify } = require('internal/util');
2527
const esmLoader = require('internal/process/esm_loader');
2628
const {
29+
ERR_INVALID_URL,
30+
ERR_INVALID_URL_SCHEME,
2731
ERR_UNKNOWN_BUILTIN_MODULE
2832
} = require('internal/errors').codes;
2933
const readFileAsync = promisify(fs.readFile);
@@ -34,6 +38,31 @@ const debug = debuglog('esm');
3438
const translators = new SafeMap();
3539
exports.translators = translators;
3640

41+
const DATA_URL_PATTERN = /^[^/]+\/[^,;]+(;base64)?,([\s\S]*)$/;
42+
function getSource(url) {
43+
const parsed = new URL(url);
44+
if (parsed.protocol === 'file:') {
45+
return readFileAsync(parsed);
46+
} else if (parsed.protocol === 'data:') {
47+
const match = DATA_URL_PATTERN.exec(parsed.pathname);
48+
if (!match) {
49+
throw new ERR_INVALID_URL(url);
50+
}
51+
const [ , base64, body ] = match;
52+
return Buffer.from(body, base64 ? 'base64' : 'utf8');
53+
} else {
54+
throw new ERR_INVALID_URL_SCHEME(['file', 'data']);
55+
}
56+
}
57+
58+
function errPath(url) {
59+
const parsed = new URL(url);
60+
if (parsed.protocol === 'file:') {
61+
return fileURLToPath(parsed);
62+
}
63+
return url;
64+
}
65+
3766
function initializeImportMeta(meta, { url }) {
3867
meta.url = url;
3968
}
@@ -45,7 +74,7 @@ async function importModuleDynamically(specifier, { url }) {
4574

4675
// Strategy for loading a standard JavaScript module
4776
translators.set('module', async function moduleStrategy(url) {
48-
const source = `${await readFileAsync(new URL(url))}`;
77+
const source = `${await getSource(url)}`;
4978
debug(`Translating StandardModule ${url}`);
5079
const { ModuleWrap, callbackMap } = internalBinding('module_wrap');
5180
const module = new ModuleWrap(stripShebang(source), url);
@@ -112,26 +141,32 @@ translators.set('builtin', async function builtinStrategy(url) {
112141
translators.set('json', async function jsonStrategy(url) {
113142
debug(`Translating JSONModule ${url}`);
114143
debug(`Loading JSONModule ${url}`);
115-
const pathname = fileURLToPath(url);
116-
const modulePath = isWindows ?
117-
StringPrototype.replace(pathname, winSepRegEx, '\\') : pathname;
118-
let module = CJSModule._cache[modulePath];
119-
if (module && module.loaded) {
120-
const exports = module.exports;
121-
return createDynamicModule([], ['default'], url, (reflect) => {
122-
reflect.exports.default.set(exports);
123-
});
144+
const pathname = url.startsWith('file:') ? fileURLToPath(url) : null;
145+
let modulePath;
146+
let module;
147+
if (pathname) {
148+
modulePath = isWindows ?
149+
StringPrototype.replace(pathname, winSepRegEx, '\\') : pathname;
150+
module = CJSModule._cache[modulePath];
151+
if (module && module.loaded) {
152+
const exports = module.exports;
153+
return createDynamicModule([], ['default'], url, (reflect) => {
154+
reflect.exports.default.set(exports);
155+
});
156+
}
124157
}
125-
const content = await readFileAsync(pathname, 'utf-8');
126-
// A require call could have been called on the same file during loading and
127-
// that resolves synchronously. To make sure we always return the identical
128-
// export, we have to check again if the module already exists or not.
129-
module = CJSModule._cache[modulePath];
130-
if (module && module.loaded) {
131-
const exports = module.exports;
132-
return createDynamicModule(['default'], url, (reflect) => {
133-
reflect.exports.default.set(exports);
134-
});
158+
const content = `${await getSource(url)}`;
159+
if (pathname) {
160+
// A require call could have been called on the same file during loading and
161+
// that resolves synchronously. To make sure we always return the identical
162+
// export, we have to check again if the module already exists or not.
163+
module = CJSModule._cache[modulePath];
164+
if (module && module.loaded) {
165+
const exports = module.exports;
166+
return createDynamicModule(['default'], url, (reflect) => {
167+
reflect.exports.default.set(exports);
168+
});
169+
}
135170
}
136171
try {
137172
const exports = JsonParse(stripBOM(content));
@@ -144,10 +179,12 @@ translators.set('json', async function jsonStrategy(url) {
144179
// parse error instead of just manipulating the original error message.
145180
// That would allow to add further properties and maybe additional
146181
// debugging information.
147-
err.message = pathname + ': ' + err.message;
182+
err.message = errPath(url) + ': ' + err.message;
148183
throw err;
149184
}
150-
CJSModule._cache[modulePath] = module;
185+
if (pathname) {
186+
CJSModule._cache[modulePath] = module;
187+
}
151188
return createDynamicModule([], ['default'], url, (reflect) => {
152189
debug(`Parsing JSONModule ${url}`);
153190
reflect.exports.default.set(module.exports);
@@ -156,14 +193,13 @@ translators.set('json', async function jsonStrategy(url) {
156193

157194
// Strategy for loading a wasm module
158195
translators.set('wasm', async function(url) {
159-
const pathname = fileURLToPath(url);
160-
const buffer = await readFileAsync(pathname);
196+
const buffer = await getSource(url);
161197
debug(`Translating WASMModule ${url}`);
162198
let compiled;
163199
try {
164200
compiled = await WebAssembly.compile(buffer);
165201
} catch (err) {
166-
err.message = pathname + ': ' + err.message;
202+
err.message = errPath(url) + ': ' + err.message;
167203
throw err;
168204
}
169205

Collapse file

‎test/es-module/test-esm-data-urls.js‎

Copy file name to clipboard
+63Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
// Flags: --experimental-modules
2+
'use strict';
3+
const common = require('../common');
4+
const assert = require('assert');
5+
function createURL(mime, body) {
6+
return `data:${mime},${body}`;
7+
}
8+
function createBase64URL(mime, body) {
9+
return `data:${mime};base64,${Buffer.from(body).toString('base64')}`;
10+
}
11+
(async () => {
12+
{
13+
const body = 'export default {a:"aaa"};';
14+
const plainESMURL = createURL('text/javascript', body);
15+
const ns = await import(plainESMURL);
16+
assert.deepStrictEqual(Object.keys(ns), ['default']);
17+
assert.deepStrictEqual(ns.default.a, 'aaa');
18+
const importerOfURL = createURL(
19+
'text/javascript',
20+
`export {default as default} from ${JSON.stringify(plainESMURL)}`
21+
);
22+
assert.strictEqual(
23+
(await import(importerOfURL)).default,
24+
ns.default
25+
);
26+
const base64ESMURL = createBase64URL('text/javascript', body);
27+
assert.notStrictEqual(
28+
await import(base64ESMURL),
29+
ns
30+
);
31+
}
32+
{
33+
const body = 'export default import.meta.url;';
34+
const plainESMURL = createURL('text/javascript', body);
35+
const ns = await import(plainESMURL);
36+
assert.deepStrictEqual(Object.keys(ns), ['default']);
37+
assert.deepStrictEqual(ns.default, plainESMURL);
38+
}
39+
{
40+
const body = '{"x": 1}';
41+
const plainESMURL = createURL('application/json', body);
42+
const ns = await import(plainESMURL);
43+
assert.deepStrictEqual(Object.keys(ns), ['default']);
44+
assert.deepStrictEqual(ns.default.x, 1);
45+
}
46+
{
47+
const body = '{"default": 2}';
48+
const plainESMURL = createURL('application/json', body);
49+
const ns = await import(plainESMURL);
50+
assert.deepStrictEqual(Object.keys(ns), ['default']);
51+
assert.deepStrictEqual(ns.default.default, 2);
52+
}
53+
{
54+
const body = 'null';
55+
const plainESMURL = createURL('invalid', body);
56+
try {
57+
await import(plainESMURL);
58+
common.mustNotCall()();
59+
} catch (e) {
60+
assert.strictEqual(e.code, 'ERR_INVALID_RETURN_PROPERTY_VALUE');
61+
}
62+
}
63+
})().then(common.mustCall());

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.