Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit fac99b8

Browse filesBrowse files
gpsheadzooba
andauthored
gh-111140: Improve PyLong_AsNativeBytes API doc example & improve the test (#115380)
This expands the examples to cover both realistic use cases for the API. I noticed thing in the test that could be done better so I added those as well: We need to guarantee that all bytes of the result are overwritten and that too many are not written. Tests now pre-fills the result with data in order to ensure that. Co-authored-by: Steve Dower <steve.dower@microsoft.com>
1 parent 7f5e3f0 commit fac99b8
Copy full SHA for fac99b8

File tree

Expand file treeCollapse file tree

2 files changed

+82
-22
lines changed
Filter options
Expand file treeCollapse file tree

2 files changed

+82
-22
lines changed

‎Doc/c-api/long.rst

Copy file name to clipboardExpand all lines: Doc/c-api/long.rst
+57-17Lines changed: 57 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -358,46 +358,86 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
358358
359359
Copy the Python integer value to a native *buffer* of size *n_bytes*::
360360
361-
int value;
362-
Py_ssize_t bytes = PyLong_AsNativeBytes(v, &value, sizeof(value), -1);
361+
int32_t value;
362+
Py_ssize_t bytes = PyLong_AsNativeBits(pylong, &value, sizeof(value), -1);
363363
if (bytes < 0) {
364-
// Error occurred
364+
// A Python exception was set with the reason.
365365
return NULL;
366366
}
367367
else if (bytes <= (Py_ssize_t)sizeof(value)) {
368368
// Success!
369369
}
370370
else {
371-
// Overflow occurred, but 'value' contains truncated value
371+
// Overflow occurred, but 'value' contains the truncated
372+
// lowest bits of pylong.
372373
}
373374
375+
The above example may look *similar* to
376+
:c:func:`PyLong_As* <PyLong_AsSize_t>`
377+
but instead fills in a specific caller defined type and never raises an
378+
error about of the :class:`int` *pylong*'s value regardless of *n_bytes*
379+
or the returned byte count.
380+
381+
To get at the entire potentially big Python value, this can be used to
382+
reserve enough space and copy it::
383+
384+
// Ask how much space we need.
385+
Py_ssize_t expected = PyLong_AsNativeBits(pylong, NULL, 0, -1);
386+
if (expected < 0) {
387+
// A Python exception was set with the reason.
388+
return NULL;
389+
}
390+
assert(expected != 0); // Impossible per the API definition.
391+
uint8_t *bignum = malloc(expected);
392+
if (!bignum) {
393+
PyErr_SetString(PyExc_MemoryError, "bignum malloc failed.");
394+
return NULL;
395+
}
396+
// Safely get the entire value.
397+
Py_ssize_t bytes = PyLong_AsNativeBits(pylong, bignum, expected, -1);
398+
if (bytes < 0) { // Exception set.
399+
free(bignum);
400+
return NULL;
401+
}
402+
else if (bytes > expected) { // Be safe, should not be possible.
403+
PyErr_SetString(PyExc_RuntimeError,
404+
"Unexpected bignum truncation after a size check.");
405+
free(bignum);
406+
return NULL;
407+
}
408+
// The expected success given the above pre-check.
409+
// ... use bignum ...
410+
free(bignum);
411+
374412
*endianness* may be passed ``-1`` for the native endian that CPython was
375413
compiled with, or ``0`` for big endian and ``1`` for little.
376414
377-
Return ``-1`` with an exception raised if *pylong* cannot be interpreted as
415+
Returns ``-1`` with an exception raised if *pylong* cannot be interpreted as
378416
an integer. Otherwise, return the size of the buffer required to store the
379417
value. If this is equal to or less than *n_bytes*, the entire value was
380-
copied.
418+
copied. ``0`` will never be returned.
381419
382-
Unless an exception is raised, all *n_bytes* of the buffer will be written
383-
with as much of the value as can fit. This allows the caller to ignore all
384-
non-negative results if the intent is to match the typical behavior of a
385-
C-style downcast. No exception is set for this case.
420+
Unless an exception is raised, all *n_bytes* of the buffer will always be
421+
written. In the case of truncation, as many of the lowest bits of the value
422+
as could fit are written. This allows the caller to ignore all non-negative
423+
results if the intent is to match the typical behavior of a C-style
424+
downcast. No exception is set on truncation.
386425
387-
Values are always copied as two's-complement, and sufficient buffer will be
426+
Values are always copied as two's-complement and sufficient buffer will be
388427
requested to include a sign bit. For example, this may cause an value that
389428
fits into 8 bytes when treated as unsigned to request 9 bytes, even though
390429
all eight bytes were copied into the buffer. What has been omitted is the
391-
zero sign bit, which is redundant when the intention is to treat the value as
392-
unsigned.
430+
zero sign bit -- redundant if the caller's intention is to treat the value
431+
as unsigned.
393432
394-
Passing zero to *n_bytes* will return the requested buffer size.
433+
Passing zero to *n_bytes* will return the size of a buffer that would
434+
be large enough to hold the value. This may be larger than technically
435+
necessary, but not unreasonably so.
395436
396437
.. note::
397438
398-
When the value does not fit in the provided buffer, the requested size
399-
returned from the function may be larger than necessary. Passing 0 to this
400-
function is not an accurate way to determine the bit length of a value.
439+
Passing *n_bytes=0* to this function is not an accurate way to determine
440+
the bit length of a value.
401441
402442
.. versionadded:: 3.13
403443

‎Lib/test/test_capi/test_long.py

Copy file name to clipboardExpand all lines: Lib/test/test_capi/test_long.py
+25-5Lines changed: 25 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -438,7 +438,12 @@ def test_long_asnativebytes(self):
438438
if support.verbose:
439439
print(f"SIZEOF_SIZE={SZ}\n{MAX_SSIZE=:016X}\n{MAX_USIZE=:016X}")
440440

441-
# These tests check that the requested buffer size is correct
441+
# These tests check that the requested buffer size is correct.
442+
# This matches our current implementation: We only specify that the
443+
# return value is a size *sufficient* to hold the result when queried
444+
# using n_bytes=0. If our implementation changes, feel free to update
445+
# the expectations here -- or loosen them to be range checks.
446+
# (i.e. 0 *could* be stored in 1 byte and 512 in 2)
442447
for v, expect in [
443448
(0, SZ),
444449
(512, SZ),
@@ -453,12 +458,25 @@ def test_long_asnativebytes(self):
453458
(-(2**256-1), 33),
454459
]:
455460
with self.subTest(f"sizeof-{v:X}"):
456-
buffer = bytearray(1)
461+
buffer = bytearray(b"\x5a")
457462
self.assertEqual(expect, asnativebytes(v, buffer, 0, -1),
458-
"PyLong_AsNativeBytes(v, NULL, 0, -1)")
463+
"PyLong_AsNativeBytes(v, <unknown>, 0, -1)")
464+
self.assertEqual(buffer, b"\x5a",
465+
"buffer overwritten when it should not have been")
459466
# Also check via the __index__ path
460467
self.assertEqual(expect, asnativebytes(Index(v), buffer, 0, -1),
461-
"PyLong_AsNativeBytes(Index(v), NULL, 0, -1)")
468+
"PyLong_AsNativeBytes(Index(v), <unknown>, 0, -1)")
469+
self.assertEqual(buffer, b"\x5a",
470+
"buffer overwritten when it should not have been")
471+
472+
# Test that we populate n=2 bytes but do not overwrite more.
473+
buffer = bytearray(b"\x99"*3)
474+
self.assertEqual(2, asnativebytes(4, buffer, 2, 0), # BE
475+
"PyLong_AsNativeBytes(v, <3 byte buffer>, 2, 0) // BE")
476+
self.assertEqual(buffer, b"\x00\x04\x99")
477+
self.assertEqual(2, asnativebytes(4, buffer, 2, 1), # LE
478+
"PyLong_AsNativeBytes(v, <3 byte buffer>, 2, 1) // LE")
479+
self.assertEqual(buffer, b"\x04\x00\x99")
462480

463481
# We request as many bytes as `expect_be` contains, and always check
464482
# the result (both big and little endian). We check the return value
@@ -510,7 +528,9 @@ def test_long_asnativebytes(self):
510528
]:
511529
with self.subTest(f"{v:X}-{len(expect_be)}bytes"):
512530
n = len(expect_be)
513-
buffer = bytearray(n)
531+
# Fill the buffer with dummy data to ensure all bytes
532+
# are overwritten.
533+
buffer = bytearray(b"\xa5"*n)
514534
expect_le = expect_be[::-1]
515535

516536
self.assertEqual(expect_n, asnativebytes(v, buffer, n, 0),

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.