You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The eval loop pays a large fixed cost on every instruction, independent of what the instruction does. In samply profiles of the release build (gc @ 2772a84e5, Apple M4), three generated metadata functions plus the dispatch body dominate self time:
workload
ExecutingFrame::run self
Opcode::deopt
Opcode::cache_entries
Instruction::as_opcode
total decode tax
fannkuch
36.8%
14.4%
9.8%
3.9%
28.1%
int while-loop
28.4%
10.6%
9.2%
4.2%
24.0%
recursive fib
14.3%
8.4%
4.3%
2.3%
15.0%
method+attr loop
8.6%
4.5%
3.9%
1.3%
9.7%
So roughly 10–28% of total runtime is spent computing per-opcode metadata that is statically known.
Causes
All in crates/vm/src/frame.rsExecutingFrame::run (~line 1513) and crates/compiler-core/src/bytecode/opcode_metadata.rs:
op.cache_entries() is called for every executed instruction (frame.rs ~1561) to skip inline-cache slots afterwards. It is implemented as match self.deoptimize() { ... } — i.e. it first runs deopt(), a ~100-arm match mapping every specialized opcode back to its base, then a second ~20-arm match. Two chained jump-table lookups per instruction, visible as the deopt/cache_entries profile entries above. CPython bakes 1 + INLINE_CACHE_ENTRIES_<OP> into each opcode handler as a compile-time constant.
lasti is an atomic touched several times per instruction: load at loop top, store in update_lasti(|i| *i += 1), re-load for lasti_before, re-load for the jump check, second store to skip caches (frame.rs ~1521–1627). Even with Relaxed ordering this forces real memory traffic and blocks the compiler from keeping the instruction pointer in a register.
Per-instruction tracing and signal checks: vm.use_tracing.get() is read twice per instruction (frame.rs ~1532, ~1565) and vm.eval_breaker_tripped() once (frame.rs ~1588), even when no tracing is active. CPython only pays the eval-breaker check on RESUME/JUMP_BACKWARD and swaps in INSTRUMENTED_* opcodes when tracing turns on, so the fast path has neither check.
Suggested direction
Make the cache skip a compile-time property of each handler: either advance lasti inside each instruction arm by a constant, or replace cache_entries() with a flat 256-byte static CACHE_ENTRIES: [u8; 256] indexed by the opcode byte (one L1 load instead of two jump tables). Same for the deopt() mapping used on the specialization/deopt paths.
Keep the instruction pointer in a local variable inside run() and publish it to the atomic lasti only where it can be observed: calls out of the frame, exception construction, trace events, safepoints. This stays free-threading-sound — other threads only need a consistent value at those boundaries.
Move tracing to instrumented opcodes (the InstrumentedLine/InstrumentedResume scaffolding already exists) and restrict the eval-breaker check to backward jumps + resume, so the straight-line fast path has zero non-dispatch loads.
Given the measured 10–28% decode tax plus the loop-baseline micro (for _ in range: acc += 1 is 63.9ns/iter vs CPython 14.0ns), this issue alone is worth a large chunk of the gap on tight loops.
Researched and written by Claude on behalf of @youknowone. Part of the performance tracking series.
Problem
The eval loop pays a large fixed cost on every instruction, independent of what the instruction does. In samply profiles of the release build (
gc@ 2772a84e5, Apple M4), three generated metadata functions plus the dispatch body dominate self time:ExecutingFrame::runselfOpcode::deoptOpcode::cache_entriesInstruction::as_opcodeSo roughly 10–28% of total runtime is spent computing per-opcode metadata that is statically known.
Causes
All in
crates/vm/src/frame.rsExecutingFrame::run(~line 1513) andcrates/compiler-core/src/bytecode/opcode_metadata.rs:op.cache_entries()is called for every executed instruction (frame.rs ~1561) to skip inline-cache slots afterwards. It is implemented asmatch self.deoptimize() { ... }— i.e. it first runsdeopt(), a ~100-arm match mapping every specialized opcode back to its base, then a second ~20-arm match. Two chained jump-table lookups per instruction, visible as thedeopt/cache_entriesprofile entries above. CPython bakes1 + INLINE_CACHE_ENTRIES_<OP>into each opcode handler as a compile-time constant.lastiis an atomic touched several times per instruction: load at loop top, store inupdate_lasti(|i| *i += 1), re-load forlasti_before, re-load for the jump check, second store to skip caches (frame.rs ~1521–1627). Even with Relaxed ordering this forces real memory traffic and blocks the compiler from keeping the instruction pointer in a register.Per-instruction tracing and signal checks:
vm.use_tracing.get()is read twice per instruction (frame.rs ~1532, ~1565) andvm.eval_breaker_tripped()once (frame.rs ~1588), even when no tracing is active. CPython only pays the eval-breaker check onRESUME/JUMP_BACKWARDand swaps inINSTRUMENTED_*opcodes when tracing turns on, so the fast path has neither check.Suggested direction
lastiinside each instruction arm by a constant, or replacecache_entries()with a flat 256-bytestatic CACHE_ENTRIES: [u8; 256]indexed by the opcode byte (one L1 load instead of two jump tables). Same for thedeopt()mapping used on the specialization/deopt paths.run()and publish it to the atomiclastionly where it can be observed: calls out of the frame, exception construction, trace events, safepoints. This stays free-threading-sound — other threads only need a consistent value at those boundaries.InstrumentedLine/InstrumentedResumescaffolding already exists) and restrict the eval-breaker check to backward jumps + resume, so the straight-line fast path has zero non-dispatch loads.Given the measured 10–28% decode tax plus the loop-baseline micro (
for _ in range: acc += 1is 63.9ns/iter vs CPython 14.0ns), this issue alone is worth a large chunk of the gap on tight loops.Researched and written by Claude on behalf of @youknowone. Part of the performance tracking series.
Part of #38.