You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: example/typed-arrays/README.md
+41-3Lines changed: 41 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -7,13 +7,16 @@ This is an extension to MessagePack which provides "native" support for JS's Typ
7
7
The official JS library can already handle TypedArrays by serialising them as binary data, but this has two disadvantages:
8
8
9
9
1. You must know, and manually construct, the correct type of array from raw binary data after deserialising.
10
-
2. The data is unaligned, which may require copying it into a new array before using it.
10
+
2. The data is unaligned, which may require copying it into a new array before using it. (See [about alignment](#about-alignment).)
11
11
12
12
Number 2 is the main reason I was inspired to write an extension to handle these types; I didn't want to give up on the possibility of zero-copy decoding.
13
13
14
14
## Spec
15
15
16
-
`data` has some internal layout which looks like the following:
16
+
TypedArray support is implemented as a MessagePack [extension](https://github.com/msgpack/msgpack/blob/master/spec.md#ext-format-family).
17
+
Extensions are encoded as a header followed by an opaque `data` array.
18
+
19
+
This extension fills `data` with an internal layout which looks like the following:
17
20
18
21
```
19
22
+--------+--------+========+========+
@@ -28,12 +31,27 @@ Where:
28
31
-`align` is a number of bytes equal to the value of `AAAAAAAA`, all of which contain 0
29
32
-`vals` is the binary content of the TypedArray
30
33
31
-
The value of `AAAAAAAA`, and therefore the number of bytes in the `align` segment, is determined so that `cont` begins on a byte offset from the _beginning of the encoded MessagePack object_ which correctly aligns `cont` for efficient access.
34
+
The value of `AAAAAAAA`, and therefore the number of bytes in the `align` segment, is determined so that `vals` begins on a byte offset from the _beginning of the encoded MessagePack object_ which correctly aligns `vals` for efficient access.
32
35
33
36
If `AAAAAAAA` is 0, then there are no `align` bytes, and `vals` begins immediately after.
34
37
35
38
Note that the length of `data`, and therefore the value of `YYYYYYYY_YYYYYYYY` includes _all_ of `artype`, `AAAAAAAA`, `align` and `vals`.
36
39
40
+
### Array types
41
+
42
+
| Constructor |`artype` decimal |`artype` hex |
43
+
| - | - | - |
44
+
| Uint8Array | 1 | 0x01 |
45
+
| Int8Array | -1 | 0xfe |
46
+
| Uint16Array | 2 | 0x02 |
47
+
| Int16Array | -2 | 0xfd |
48
+
| Uint32Array | 3 | 0x03 |
49
+
| Int32Array | -3 | 0xfc |
50
+
| BigUint64Array | 4 | 0x04 |
51
+
| BigInt64Array | -4 | 0xfb |
52
+
| Float32Array | 9 | 0x09 |
53
+
| Float64Array | 10 | 0x0a |
54
+
37
55
## Example
38
56
39
57
A Float32Array containing 10 values will have a `data` size starting at 42 bytes if there is no alignment:
@@ -86,3 +104,23 @@ Where:
86
104
-`0x03` is the number of alignment bytes
87
105
- 3 zeros are required for alignment
88
106
-`vals` contains the actual floating-point data
107
+
108
+
## About alignment
109
+
110
+
This [SO question](https://stackoverflow.com/q/7372124) demonstrates the problem:
111
+
112
+
```js
113
+
newFloat32Array(buffer, 31, 6);
114
+
```
115
+
116
+
will throw an exception.
117
+
When creating any TypedArray, the offset (2nd argument) must be a multiple of the byte length of the element type.
118
+
In the case of a Float32Array, 31 is not a multiple of 4 so the creation fails.
119
+
120
+
As the top answer states,
121
+
122
+
> Some architectures do not allow unaligned word accesses, and there are performance penalties on architectures that do allow it such as x86 (though some instructions must be aligned).
123
+
124
+
[This post](http://www.songho.ca/misc/alignment/dataalign.html) contains more details.
125
+
So the typical approach if you receive some data from a MessagePack buffer which you want to access as a TypedArray is to copy the data out into a new buffer entirely.
126
+
Because new buffers are correctly aligned (e.g. their first byte falls on a [max_align_t](https://en.cppreference.com/w/c/types/max_align_t) memory address), and the offset will be 0 for the new buffer, your access will work fine.
0 commit comments