Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 972fba7

Browse filesBrowse files
gfxgfx
committed
doc
1 parent 058e90d commit 972fba7
Copy full SHA for 972fba7

File tree

Expand file treeCollapse file tree

1 file changed

+99
-56
lines changed
Filter options
Expand file treeCollapse file tree

1 file changed

+99
-56
lines changed

‎README.md

Copy file name to clipboardExpand all lines: README.md
+99-56Lines changed: 99 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ deepStrictEqual(decode(encoded), object);
5353
- [ExtensionCodec context](#extensioncodec-context)
5454
- [Handling BigInt with ExtensionCodec](#handling-bigint-with-extensioncodec)
5555
- [The temporal module as timestamp extensions](#the-temporal-module-as-timestamp-extensions)
56+
- [Faster way to decode a large array of floating point numbers](#faster-way-to-decode-a-large-array-of-floating-point-numbers)
5657
- [Decoding a Blob](#decoding-a-blob)
5758
- [MessagePack Specification](#messagepack-specification)
5859
- [MessagePack Mapping Table](#messagepack-mapping-table)
@@ -109,17 +110,17 @@ console.log(buffer);
109110

110111
#### `EncoderOptions`
111112

112-
Name|Type|Default
113-
----|----|----
114-
extensionCodec | ExtensionCodec | `ExtensionCodec.defaultCodec`
115-
context | user-defined | -
116-
useBigInt64 | boolean | false
117-
maxDepth | number | `100`
118-
initialBufferSize | number | `2048`
119-
sortKeys | boolean | false
120-
forceFloat32 | boolean | false
121-
forceIntegerToFloat | boolean | false
122-
ignoreUndefined | boolean | false
113+
| Name | Type | Default |
114+
| ------------------- | -------------- | ----------------------------- |
115+
| extensionCodec | ExtensionCodec | `ExtensionCodec.defaultCodec` |
116+
| context | user-defined | - |
117+
| useBigInt64 | boolean | false |
118+
| maxDepth | number | `100` |
119+
| initialBufferSize | number | `2048` |
120+
| sortKeys | boolean | false |
121+
| forceFloat32 | boolean | false |
122+
| forceIntegerToFloat | boolean | false |
123+
| ignoreUndefined | boolean | false |
123124

124125
### `decode(buffer: ArrayLike<number> | BufferSource, options?: DecoderOptions): unknown`
125126

@@ -143,17 +144,17 @@ NodeJS `Buffer` is also acceptable because it is a subclass of `Uint8Array`.
143144

144145
#### `DecoderOptions`
145146

146-
Name|Type|Default
147-
----|----|----
148-
extensionCodec | ExtensionCodec | `ExtensionCodec.defaultCodec`
149-
context | user-defined | -
150-
useBigInt64 | boolean | false
151-
rawStrings | boolean | false
152-
maxStrLength | number | `4_294_967_295` (UINT32_MAX)
153-
maxBinLength | number | `4_294_967_295` (UINT32_MAX)
154-
maxArrayLength | number | `4_294_967_295` (UINT32_MAX)
155-
maxMapLength | number | `4_294_967_295` (UINT32_MAX)
156-
maxExtLength | number | `4_294_967_295` (UINT32_MAX)
147+
| Name | Type | Default |
148+
| -------------- | -------------- | ----------------------------- |
149+
| extensionCodec | ExtensionCodec | `ExtensionCodec.defaultCodec` |
150+
| context | user-defined | - |
151+
| useBigInt64 | boolean | false |
152+
| rawStrings | boolean | false |
153+
| maxStrLength | number | `4_294_967_295` (UINT32_MAX) |
154+
| maxBinLength | number | `4_294_967_295` (UINT32_MAX) |
155+
| maxArrayLength | number | `4_294_967_295` (UINT32_MAX) |
156+
| maxMapLength | number | `4_294_967_295` (UINT32_MAX) |
157+
| maxExtLength | number | `4_294_967_295` (UINT32_MAX) |
157158

158159
To skip UTF-8 decoding of strings, `rawStrings` can be set to `true`. In this case, strings are decoded into `Uint8Array`.
159160

@@ -454,6 +455,48 @@ deepStrictEqual(decoded, instant);
454455

455456
This will become default in this library with major-version increment, if the temporal module is standardized.
456457

458+
## Faster way to decode a large array of floating point numbers
459+
460+
If there are large arrays of floating point numbers in your payload, there
461+
is a way to decode it faster: define a custom extension type for `Float#Array`
462+
with alignment.
463+
464+
An extension type's `encode` method can return a function that takes a parameter
465+
`pos: number`. This parameter can be used to make alignment of the buffer,
466+
resulting decoding it much more performant.
467+
468+
See an example implementation for `Float32Array`:
469+
470+
```typescript
471+
const extensionCodec = new ExtensionCodec();
472+
473+
const EXT_TYPE_FLOAT32ARRAY = 0; // Any in 0-127
474+
extensionCodec.register({
475+
type: EXT_TYPE_FLOAT32ARRAY,
476+
encode: (object: unknown) => {
477+
if (object instanceof Float32Array) {
478+
return (pos: number) => {
479+
const bpe = Float32Array.BYTES_PER_ELEMENT;
480+
const padding = 1 + ((bpe - ((pos + 1) % bpe)) % bpe);
481+
const data = new Uint8Array(object.buffer);
482+
const result = new Uint8Array(padding + data.length);
483+
result[0] = padding;
484+
result.set(data, padding);
485+
return result;
486+
};
487+
}
488+
return null;
489+
},
490+
decode: (data: Uint8Array) => {
491+
const padding = data[0]!;
492+
const bpe = Float32Array.BYTES_PER_ELEMENT;
493+
const offset = data.byteOffset + padding;
494+
const length = data.byteLength - padding;
495+
return new Float32Array(data.buffer, offset, length / bpe);
496+
},
497+
});
498+
```
499+
457500
## Decoding a Blob
458501

459502
[`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob) is a binary data container provided by browsers. To read its contents when it contains a MessagePack binary, you can use `Blob#arrayBuffer()` or `Blob#stream()`. `Blob#stream()`
@@ -495,18 +538,18 @@ The mapping of integers varies on the setting of `useBigInt64`.
495538

496539
The default, `useBigInt64: false` is:
497540

498-
Source Value|MessagePack Format|Value Decoded
499-
----|----|----
500-
null, undefined|nil|null (*1)
501-
boolean (true, false)|bool family|boolean (true, false)
502-
number (53-bit int)|int family|number
503-
number (64-bit float)|float family|number
504-
string|str family|string (*2)
505-
ArrayBufferView |bin family|Uint8Array (*3)
506-
Array|array family|Array
507-
Object|map family|Object (*4)
508-
Date|timestamp ext family|Date (*5)
509-
bigint|N/A|N/A (*6)
541+
| Source Value | MessagePack Format | Value Decoded |
542+
| --------------------- | -------------------- | --------------------- |
543+
| null, undefined | nil | null (*1) |
544+
| boolean (true, false) | bool family | boolean (true, false) |
545+
| number (53-bit int) | int family | number |
546+
| number (64-bit float) | float family | number |
547+
| string | str family | string (*2) |
548+
| ArrayBufferView | bin family | Uint8Array (*3) |
549+
| Array | array family | Array |
550+
| Object | map family | Object (*4) |
551+
| Date | timestamp ext family | Date (*5) |
552+
| bigint | N/A | N/A (*6) |
510553

511554
* *1 Both `null` and `undefined` are mapped to `nil` (`0xC0`) type, and are decoded into `null`
512555
* *2 If you'd like to skip UTF-8 decoding of strings, set `rawStrings: true`. In this case, strings are decoded into `Uint8Array`.
@@ -517,18 +560,18 @@ bigint|N/A|N/A (*6)
517560

518561
If you set `useBigInt64: true`, the following mapping is used:
519562

520-
Source Value|MessagePack Format|Value Decoded
521-
----|----|----
522-
null, undefined|nil|null
523-
boolean (true, false)|bool family|boolean (true, false)
524-
**number (32-bit int)**|int family|number
525-
**number (except for the above)**|float family|number
526-
**bigint**|int64 / uint64|bigint (*7)
527-
string|str family|string
528-
ArrayBufferView |bin family|Uint8Array
529-
Array|array family|Array
530-
Object|map family|Object
531-
Date|timestamp ext family|Date
563+
| Source Value | MessagePack Format | Value Decoded |
564+
| --------------------------------- | -------------------- | --------------------- |
565+
| null, undefined | nil | null |
566+
| boolean (true, false) | bool family | boolean (true, false) |
567+
| **number (32-bit int)** | int family | number |
568+
| **number (except for the above)** | float family | number |
569+
| **bigint** | int64 / uint64 | bigint (*7) |
570+
| string | str family | string |
571+
| ArrayBufferView | bin family | Uint8Array |
572+
| Array | array family | Array |
573+
| Object | map family | Object |
574+
| Date | timestamp ext family | Date |
532575

533576

534577
* *7 If the bigint is larger than the max value of uint64 or smaller than the min value of int64, then the behavior is undefined.
@@ -570,16 +613,16 @@ However, MessagePack can handles binary data effectively, actual performance dep
570613

571614
Benchmark on NodeJS/v22.13.1 (V8/12.4)
572615

573-
operation | op | ms | op/s
574-
----------------------------------------------------------------- | ------: | ----: | ------:
575-
buf = Buffer.from(JSON.stringify(obj)); | 1348700 | 5000 | 269740
576-
obj = JSON.parse(buf.toString("utf-8")); | 1700300 | 5000 | 340060
577-
buf = require("msgpack-lite").encode(obj); | 591300 | 5000 | 118260
578-
obj = require("msgpack-lite").decode(buf); | 539500 | 5000 | 107900
579-
buf = require("@msgpack/msgpack").encode(obj); | 1238700 | 5000 | 247740
580-
obj = require("@msgpack/msgpack").decode(buf); | 1402000 | 5000 | 280400
581-
buf = /* @msgpack/msgpack */ encoder.encode(obj); | 1379800 | 5000 | 275960
582-
obj = /* @msgpack/msgpack */ decoder.decode(buf); | 1406100 | 5000 | 281220
616+
| operation | op | ms | op/s |
617+
| ------------------------------------------------- | ------: | ---: | -----: |
618+
| buf = Buffer.from(JSON.stringify(obj)); | 1348700 | 5000 | 269740 |
619+
| obj = JSON.parse(buf.toString("utf-8")); | 1700300 | 5000 | 340060 |
620+
| buf = require("msgpack-lite").encode(obj); | 591300 | 5000 | 118260 |
621+
| obj = require("msgpack-lite").decode(buf); | 539500 | 5000 | 107900 |
622+
| buf = require("@msgpack/msgpack").encode(obj); | 1238700 | 5000 | 247740 |
623+
| obj = require("@msgpack/msgpack").decode(buf); | 1402000 | 5000 | 280400 |
624+
| buf = /* @msgpack/msgpack */ encoder.encode(obj); | 1379800 | 5000 | 275960 |
625+
| obj = /* @msgpack/msgpack */ decoder.decode(buf); | 1406100 | 5000 | 281220 |
583626

584627
Note that `JSON` cases use `Buffer` to emulate I/O where a JavaScript string must be converted into a byte array encoded in UTF-8, whereas MessagePack modules deal with byte arrays.
585628

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.