Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit ef774ff

Browse filesBrowse files
kthelgasonrvagg
authored andcommitted
zlib: add support for concatenated members
According to the spec gzipped archives can contain more than one compressed member. Previously Node's gzip implementation would only unzip the first member and throw away the rest of the compressed data. Issue #4306 is an example of this occurring in daily use. Fixes: #4306 PR-URL: #5120 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
1 parent b16f67a commit ef774ff
Copy full SHA for ef774ff

File tree

Expand file treeCollapse file tree

3 files changed

+86
-0
lines changed
Open diff view settings
Filter options
Expand file treeCollapse file tree

3 files changed

+86
-0
lines changed
Open diff view settings
Collapse file

‎src/node_zlib.cc‎

Copy file name to clipboardExpand all lines: src/node_zlib.cc
+18Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,9 @@ enum node_zlib_mode {
4141
UNZIP
4242
};
4343

44+
#define GZIP_HEADER_ID1 0x1f
45+
#define GZIP_HEADER_ID2 0x8b
46+
#define GZIP_MIN_HEADER_SIZE 10
4447

4548
void InitZlib(v8::Local<v8::Object> target);
4649

@@ -254,6 +257,19 @@ class ZCtx : public AsyncWrap {
254257
ctx->err_ = Z_NEED_DICT;
255258
}
256259
}
260+
while (ctx->strm_.avail_in >= GZIP_MIN_HEADER_SIZE &&
261+
ctx->mode_ == GUNZIP) {
262+
// Bytes remain in input buffer. Perhaps this is another compressed
263+
// member in the same archive, or just trailing garbage.
264+
// Check the header to find out.
265+
if (ctx->strm_.next_in[0] != GZIP_HEADER_ID1 ||
266+
ctx->strm_.next_in[1] != GZIP_HEADER_ID2) {
267+
// Not a valid gzip member
268+
break;
269+
}
270+
Reset(ctx);
271+
ctx->err_ = inflate(&ctx->strm_, ctx->flush_);
272+
}
257273
break;
258274
default:
259275
CHECK(0 && "wtf?");
@@ -524,10 +540,12 @@ class ZCtx : public AsyncWrap {
524540
switch (ctx->mode_) {
525541
case DEFLATE:
526542
case DEFLATERAW:
543+
case GZIP:
527544
ctx->err_ = deflateReset(&ctx->strm_);
528545
break;
529546
case INFLATE:
530547
case INFLATERAW:
548+
case GUNZIP:
531549
ctx->err_ = inflateReset(&ctx->strm_);
532550
break;
533551
default:
Collapse file
+18Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
'use strict';
2+
// Test unzipping a gzip file that contains multiple concatenated "members"
3+
4+
const common = require('../common');
5+
const assert = require('assert');
6+
const zlib = require('zlib');
7+
8+
const data = Buffer.concat([
9+
zlib.gzipSync('abc'),
10+
zlib.gzipSync('def')
11+
]);
12+
13+
assert.equal(zlib.gunzipSync(data).toString(), 'abcdef');
14+
15+
zlib.gunzip(data, common.mustCall((err, result) => {
16+
assert.ifError(err);
17+
assert.equal(result, 'abcdef', 'result should match original string');
18+
}));
Collapse file
+50Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
'use strict';
2+
// test unzipping a gzip file that has trailing garbage
3+
4+
const common = require('../common');
5+
const assert = require('assert');
6+
const zlib = require('zlib');
7+
8+
// should ignore trailing null-bytes
9+
let data = Buffer.concat([
10+
zlib.gzipSync('abc'),
11+
zlib.gzipSync('def'),
12+
Buffer(10).fill(0)
13+
]);
14+
15+
assert.equal(zlib.gunzipSync(data).toString(), 'abcdef');
16+
17+
zlib.gunzip(data, common.mustCall((err, result) => {
18+
assert.ifError(err);
19+
assert.equal(result, 'abcdef', 'result should match original string');
20+
}));
21+
22+
// if the trailing garbage happens to look like a gzip header, it should
23+
// throw an error.
24+
data = Buffer.concat([
25+
zlib.gzipSync('abc'),
26+
zlib.gzipSync('def'),
27+
Buffer([0x1f, 0x8b, 0xff, 0xff]),
28+
Buffer(10).fill(0)
29+
]);
30+
31+
assert.throws(() => zlib.gunzipSync(data));
32+
33+
zlib.gunzip(data, common.mustCall((err, result) => {
34+
assert(err);
35+
}));
36+
37+
// In this case the trailing junk is too short to be a gzip segment
38+
// So we ignore it and decompression succeeds.
39+
data = Buffer.concat([
40+
zlib.gzipSync('abc'),
41+
zlib.gzipSync('def'),
42+
Buffer([0x1f, 0x8b, 0xff, 0xff])
43+
]);
44+
45+
assert.equal(zlib.gunzipSync(data).toString(), 'abcdef');
46+
47+
zlib.gunzip(data, common.mustCall((err, result) => {
48+
assert.ifError(err);
49+
assert.equal(result, 'abcdef', 'result should match original string');
50+
}));

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.