Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

json: Fast path for string encoding #133239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
Loading
from
Open

json: Fast path for string encoding #133239

wants to merge 2 commits into from
+204 −31

Conversation

nineteendo
Copy link
Contributor

@nineteendo nineteendo commented May 1, 2025 •
edited
Loading

pyperformance (with --enable-optimizations and --with-lto)

main.json
=========

Performance version: 1.11.0
Python version: 3.15.0a0 (64-bit) revision fe9f6e829a
Report on macOS-13.7.6-x86_64-i386-64bit-Mach-O
Number of logical CPUs: 8
Start date: 2025-05-15 08:56:50.561560
End date: 2025-05-15 08:57:59.544685

feature.json
============

Performance version: 1.11.0
Python version: 3.15.0a0 (64-bit) revision 32eefb3ef2
Report on macOS-13.7.6-x86_64-i386-64bit-Mach-O
Number of logical CPUs: 8
Start date: 2025-05-15 08:54:12.863957
End date: 2025-05-15 08:55:18.964946

### json_dumps ###
Mean +- std dev: 12.3 ms +- 0.2 ms -> 10.4 ms +- 0.2 ms: 1.19x faster
Significant (t=68.04)

jsonyx-performance-tests (with --enable-optimizations and --with-lto)

encode main feature improvement
Dict with 65,536 booleans 9913.14 μs 5937.37 μs 1.67x faster
List of 65,536 ASCII strings 12921.10 μs 6773.93 μs 1.91x faster
List of 65,536 strings 86405.36 μs 114756.66 μs 1.33x slower

@nineteendo nineteendo mentioned this pull request May 1, 2025
Open
@methane
Copy link
Member

methane commented May 1, 2025 •
edited
Loading

https://gist.github.com/methane/e080ec9783db2a313f40a2b9e1837e72

Benchmark main #133186 #133239
json_dumps: List of 256 booleans 16.6 us not significant 17.2 us: 1.03x slower
json_dumps: List of 256 ASCII strings 67.9 us 34.7 us: 1.96x faster 46.5 us: 1.46x faster
json_dumps: List of 256 dicts with 1 int 122 us 101 us: 1.21x faster 112 us: 1.09x faster
json_dumps: Medium complex object 205 us 173 us: 1.18x faster 189 us: 1.09x faster
json_dumps: List of 256 strings 330 us 302 us: 1.09x faster 298 us: 1.11x faster
json_dumps: Complex object 2.57 ms 1.96 ms: 1.31x faster not significant
json_dumps: Dict with 256 lists of 256 dicts with 1 int 30.5 ms 26.5 ms: 1.15x faster 29.4 ms: 1.04x faster
json_dumps(ensure_ascii=False): List of 256 booleans 16.6 us not significant 17.2 us: 1.03x slower
json_dumps(ensure_ascii=False): List of 256 ASCII strings 68.1 us 34.6 us: 1.96x faster 46.5 us: 1.46x faster
json_dumps(ensure_ascii=False): List of 256 dicts with 1 int 122 us 101 us: 1.21x faster 112 us: 1.09x faster
json_dumps(ensure_ascii=False): Medium complex object 205 us 172 us: 1.19x faster 188 us: 1.09x faster
json_dumps(ensure_ascii=False): List of 256 strings 329 us 303 us: 1.09x faster 298 us: 1.11x faster
json_dumps(ensure_ascii=False): Complex object 2.56 ms 1.95 ms: 1.31x faster not significant
json_dumps(ensure_ascii=False): Dict with 256 lists of 256 dicts with 1 int 30.6 ms 26.5 ms: 1.15x faster 29.4 ms: 1.04x faster
json_loads: List of 256 floats 91.4 us 88.3 us: 1.03x faster not significant
json_loads: List of 256 strings 848 us 816 us: 1.04x faster not significant
Geometric mean (ref) 1.13x faster 1.05x faster

Benchmark hidden because not significant (10): json_dumps: List of 256 floats, json_dumps(ensure_ascii=False): List of 256 floats, json_loads: List of 256 booleans, json_loads: List of 256 ASCII strings, json_loads: List of 256 dicts with 1 int, json_loads: Medium complex object, json_loads: Complex object, json_loads: Dict with 256 lists of 256 dicts with 1 int, json_loads: List of 256 stringsensure_ascii=False, json_loads: Complex objectensure_ascii=False

@nineteendo
Copy link
Contributor Author

@mdboom do you have the results of the Faster CPython infrastructure?

@mdboom
Copy link
Contributor

mdboom commented May 10, 2025

@mdboom do you have the results of the Faster CPython infrastructure?

Sorry, forgot to come back to them.

They are here: https://github.com/faster-cpython/benchmarking-public/blob/main/results/bm-20250501-3.14.0a7%2B-930e938/bm-20250501-linux-x86_64-nineteendo-speedup_json_encode-3.14.0a7%2B-930e938-vs-base.svg

Confirmed 14% faster on json_dumps benchmark. In the noise for the others (as one would expect).

@nineteendo nineteendo marked this pull request as ready for review May 13, 2025 14:06
nineteendo
nineteendo commented May 15, 2025
View reviewed changes
Modules/_json.c
{
/* Return the JSON representation of a string */
PyObject *encoded;

if (s->fast_encode) {
return s->fast_encode(NULL, obj);
if (PyUnicodeWriter_WriteChar(writer, '"') < 0) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to avoid this call for the normal path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
awaiting review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nineteendo @methane @mdboom
Morty Proxy This is a proxified and sanitized view of the page, visit original site.