Problem
Looping over the standard iterator adapters — zip, enumerate, map, dict views — costs 75–140 ns per item on top of the loop baseline, where CPython pays 3–11 ns. Each adapter allocates one or more fresh heap objects per item and goes through generic dispatch, where CPython reuses its result tuple and keeps plain machine-word state.
Measurements
Apple M4 (10 cores), macOS Darwin 25.5.0. RustPython typelock @ 8d7d9d9, cargo build --release; CPython 3.14.2 (Homebrew). 2026-07-23. Min-of-3 alternating passes; incremental = per-item cost minus the acc += 1 loop baseline (4.0x).
| benchmark |
ratio |
incremental RustPython |
incremental CPython |
for x in map(int, r) |
8.6x |
+135 ns/item |
+5 ns |
for a, b in zip(r, r) |
7.2x |
+142 ns/item |
+11 ns |
for i, v in enumerate(r) |
6.8x |
+117 ns/item |
+9 ns |
for k, v in d.items() |
6.7x |
+75 ns/item |
+3 ns |
control: for x in lst |
4.1x |
|
|
control: for x in tup |
4.0x |
|
|
The controls show plain list/tuple iteration (which has ForIterList/ForIterTuple specializations) already sits at the interpreter baseline — the adapter gap is in the adapters themselves, not FOR_ITER.
Mechanism
zip (crates/vm/src/builtins/zip.rs:80-113): per item, let mut next_objs = Vec::new() + push per sub-iterator (heap alloc + possible growth realloc), then vm.ctx.new_tuple(next_objs) (into_boxed_slice shrink-realloc + PyRef alloc). 2–3 allocator interactions per item, no result reuse.
enumerate (crates/vm/src/builtins/enumerate.rs:20-24,80-86): the counter is a PyRwLock<BigInt>. Per item: write-lock acquire, counter.clone() (BigInt clone), bigint += 1, then (position, next_obj).to_pyobject(vm) allocates a fresh PyInt and a fresh 2-tuple.
map (crates/vm/src/builtins/map.rs:88-123): per item, Vec::new() + push per sub-iterator, then zelf.mapper.call(next_objs, vm) — an args Vec + FuncArgs construction through the generic call slot for every mapped element (the callee's own call cost is the CALL-family issue).
All of these also reach the adapter through the generic PyIter::next slot dispatch per item.
What CPython does
zip_next (Python/bltinmodule.c:3169): when the result tuple is uniquely referenced (the common for-loop case) it recycles the same tuple in place — zero allocation per item.
enum_next (Objects/enumobject.c:238-265): en_index is a plain C ssize_t (bigint fallback only at PY_SSIZE_T_MAX); small indices come from the immortal small-int table; the result tuple is reused the same way.
map_next: _PyObject_VectorcallTstate with a stack-allocated small_stack array — no heap allocation for arguments.
Suggested direction
- Result-tuple reuse for
zip/enumerate/dict-view iterators: recycle the previous result tuple when it is uniquely referenced. Under free-threading this is safe with the same check CPython 3.14t uses (_PyObject_IsUniquelyReferenced — refcount==1 observed by the owning thread; adapters are typically single-consumer, and a shared adapter still falls back to fresh allocation).
enumerate counter → usize (atomic or lock-free) with bigint fallback only on overflow / big start=; produce the index via the small-int cache / normal int path (per-item int allocation itself is the Perf: small int/float arithmetic allocates heap objects; specialized BinaryOp*Int skips the i64 fast path #43 axis).
map argument passing via a small inline buffer instead of Vec + FuncArgs heap churn — same fix shape as the native-call capstone (borrowed arg slice), so these can share infrastructure.
- The same tuple-reuse treatment applies to
dict.items() iteration (per-item 2-tuple).
Cross-references: per-item PyInt allocation → #43; atomic refcount churn on each element → #44.
Part of #38.
Researched and written by Claude on behalf of @youknowone. Part of the performance tracking series.
Problem
Looping over the standard iterator adapters —
zip,enumerate,map, dict views — costs 75–140 ns per item on top of the loop baseline, where CPython pays 3–11 ns. Each adapter allocates one or more fresh heap objects per item and goes through generic dispatch, where CPython reuses its result tuple and keeps plain machine-word state.Measurements
Apple M4 (10 cores), macOS Darwin 25.5.0. RustPython
typelock@ 8d7d9d9,cargo build --release; CPython 3.14.2 (Homebrew). 2026-07-23. Min-of-3 alternating passes; incremental = per-item cost minus theacc += 1loop baseline (4.0x).for x in map(int, r)for a, b in zip(r, r)for i, v in enumerate(r)for k, v in d.items()for x in lstfor x in tupThe controls show plain list/tuple iteration (which has
ForIterList/ForIterTuplespecializations) already sits at the interpreter baseline — the adapter gap is in the adapters themselves, not FOR_ITER.Mechanism
zip(crates/vm/src/builtins/zip.rs:80-113): per item,let mut next_objs = Vec::new()+ push per sub-iterator (heap alloc + possible growth realloc), thenvm.ctx.new_tuple(next_objs)(into_boxed_sliceshrink-realloc + PyRef alloc). 2–3 allocator interactions per item, no result reuse.enumerate(crates/vm/src/builtins/enumerate.rs:20-24,80-86): the counter is aPyRwLock<BigInt>. Per item: write-lock acquire,counter.clone()(BigInt clone), bigint+= 1, then(position, next_obj).to_pyobject(vm)allocates a freshPyIntand a fresh 2-tuple.map(crates/vm/src/builtins/map.rs:88-123): per item,Vec::new()+ push per sub-iterator, thenzelf.mapper.call(next_objs, vm)— an argsVec+FuncArgsconstruction through the generic call slot for every mapped element (the callee's own call cost is the CALL-family issue).All of these also reach the adapter through the generic
PyIter::nextslot dispatch per item.What CPython does
zip_next(Python/bltinmodule.c:3169): when the result tuple is uniquely referenced (the common for-loop case) it recycles the same tuple in place — zero allocation per item.enum_next(Objects/enumobject.c:238-265):en_indexis a plain Cssize_t(bigint fallback only atPY_SSIZE_T_MAX); small indices come from the immortal small-int table; the result tuple is reused the same way.map_next:_PyObject_VectorcallTstatewith a stack-allocatedsmall_stackarray — no heap allocation for arguments.Suggested direction
zip/enumerate/dict-view iterators: recycle the previous result tuple when it is uniquely referenced. Under free-threading this is safe with the same check CPython 3.14t uses (_PyObject_IsUniquelyReferenced— refcount==1 observed by the owning thread; adapters are typically single-consumer, and a shared adapter still falls back to fresh allocation).enumeratecounter →usize(atomic or lock-free) with bigint fallback only on overflow / bigstart=; produce the index via the small-int cache / normal int path (per-item int allocation itself is the Perf: small int/float arithmetic allocates heap objects; specialized BinaryOp*Int skips the i64 fast path #43 axis).mapargument passing via a small inline buffer instead ofVec+FuncArgsheap churn — same fix shape as the native-call capstone (borrowed arg slice), so these can share infrastructure.dict.items()iteration (per-item 2-tuple).Cross-references: per-item
PyIntallocation → #43; atomic refcount churn on each element → #44.Part of #38.
Researched and written by Claude on behalf of @youknowone. Part of the performance tracking series.