Find out whether a learned decoder actually beats minimum-weight perfect matching on your noise model — and what it costs you in latency.

Quantum error correction is the gate between today's noisy processors and useful quantum computing, and the decoder is the classical software sitting in its critical path. Since Google's Willow results and DeepMind's AlphaQubit, "replace the matching decoder with a neural network" has become one of the most active ideas in the field — and one of the easiest to get wrong, because a neural decoder that wins on accuracy while missing the real-time latency budget is not a decoder, it is a paper.

This repository is the harness that settles the question on your code parameters and your noise model: it generates syndromes, runs an MWPM baseline and a trained neural decoder over the identical shots, and reports accuracy and latency side by side.

• The problem it solves
• Who it's for
• Architecture
• Install
• Quick start
• The five stages
• Use it as a library
• Bring your own decoder
• Results
• Why this stack
• Project layout
• Operational characteristics
• Testing
• Honest disclaimer
• References
• Citing this work
• Contributing

A surface-code patch emits a stream of stabiliser measurements. Errors show up as "detection events", and the decoder's job is to infer, from those events alone, whether the logical qubit has flipped. Get it wrong and the computation is silently corrupt.

Minimum-weight perfect matching is the incumbent: fast, well understood, and provably good on graph-like noise. But it is built from an assumed error model. Where the real device deviates — leakage, crosstalk, correlated errors — a decoder trained on actual syndrome data can in principle do better.

"In principle" is doing a lot of work in that sentence. Answering it for a specific setup requires a fair comparison, and a fair comparison requires infrastructure: identical shots for both decoders, enough of them that the confidence interval is smaller than the effect, and honest latency measurement. That infrastructure is what this repo is.

flowchart TB
    subgraph spec["Experiment definition"]
        CS["CodeSpec<br/>distance · rounds · p · basis"]
    end

    subgraph gen["Syndrome generation"]
        ST["stim<br/>noisy stabiliser circuit"]
        DEM["Detector error model<br/>(decomposed)"]
        SAMP["Detector sampler"]
        PQ[("Parquet shards<br/>partitioned by slug")]
    end

    subgraph dec["Decoders"]
        MWPM["PyMatching<br/>MWPM baseline"]
        NN["Neural decoder<br/>PyTorch MLP"]
    end

    subgraph out["Evaluation"]
        EV["Paired evaluation<br/>identical shots"]
        RES["Logical error rate · Λ<br/>latency p50/p95 · Wilson CI"]
        LOG[["events.jsonl"]]
    end

    CS --> ST --> SAMP --> PQ
    ST --> DEM --> MWPM
    PQ --> NN
    PQ --> EV
    MWPM --> EV
    NN --> EV
    EV --> RES
    EV --> LOG

The key structural decision: the detector error model feeds the baseline, and the sampled shots feed the neural decoder, but both are evaluated on the same held-out shots. Anything else is not a comparison.

Requires Python 3.11 and no quantum hardware — everything runs on a laptop CPU.

git clone git@github.com:ghoshp83/qec-decoder-lab.git
cd qec-decoder-lab

uv venv --python 3.11
uv pip install --index-url https://download.pytorch.org/whl/cpu torch
uv pip install -e ".[dev]"

The CPU wheel of PyTorch is pulled from its own index first; everything else comes from PyPI. Confirm the install with uv run qeclab version.

One command samples, trains and evaluates, printing the verdict:

uv run qeclab run --distance 5 --rounds 5 --noise 0.005 --shots 200000

d5_r5_p0.005_z  (40000 held-out shots)
  mwpm       LER 0.01440 [0.01328, 0.01561]  576/40000 errors  p95 43.0us/shot
  mlp        LER 0.08217 [0.07952, 0.08491]  3287/40000 errors  p95 4188.0us/shot
verdict: MWPM significantly better (non-overlapping 95% intervals)

Add --json-out result.json for machine-readable output that a pipeline can consume.

run is a convenience wrapper over five verbs you can also drive separately — useful because sampling is the expensive step and you rarely want to repeat it while iterating on a decoder.

# 1+2. Sample syndromes, then decode them with the matching baseline
uv run qeclab sample   --distance 5 --rounds 5 --noise 0.005 --shots 200000
uv run qeclab baseline --distance 5 --rounds 5 --noise 0.005

# 3+4. Train a neural decoder on those shots and compare on held-out data
uv run qeclab train    --distance 5 --rounds 5 --noise 0.005 --epochs 20
uv run qeclab evaluate --distance 5 --rounds 5 --noise 0.005

# 5. Sweep the physical error rate to locate the threshold
uv run qeclab threshold --distances 3 5 7 --noise-range 0.002 0.012 --points 6 \
    --svg-out docs/threshold.svg

Every command appends structured JSON events to runs/<run_id>/events.jsonl, so any published number is traceable to the run that produced it.

The five stages are also a curated Python API — import from the top-level package and drive the pipeline directly:

from qecdecoder import CodeSpec, sample_shots, make_splits, train_decoder, compare_decoders

spec = CodeSpec(distance=5, rounds=5, noise=0.005)
detections, observables = sample_shots(spec, shots=200_000, seed=0)
splits = make_splits(detections, observables)

decoder, history = train_decoder(splits)
result = compare_decoders(spec, splits.test.detections, splits.test.observables, decoder)
print(result.summary())

The package ships a py.typed marker, so your type-checker sees the full annotations.

The whole point of the harness is to benchmark a new decoder, so the decoder interface is a formal, runtime_checkable protocol — two members, no base class to inherit:

import numpy as np
from qecdecoder import CodeSpec, sample_shots, compare_decoders

class MyDecoder:
    name = "my-decoder"

    def decode_batch(self, detections: np.ndarray) -> np.ndarray:
        # detections: (shots, num_detectors) bool/uint8
        # return:     (shots,) predicted logical-observable flips
        ...

spec = CodeSpec(distance=5, rounds=5, noise=0.005)
detections, observables = sample_shots(spec, shots=200_000, seed=0)

comparison = compare_decoders(spec, detections, observables, MyDecoder())
print(comparison.verdict)  # baseline_better | neural_better | inconclusive

compare_decoders scores your decoder and the MWPM baseline on the same shots and returns a three-way verdict with Wilson confidence intervals — you get a correct, honest comparison without touching the sampling, scoring or significance plumbing.

Measured on one CPU core; reproduce with the commands above. Every number here came out of the CLI — none are hand-written.

Distance 5, 5 rounds, p = 0.005, 200k shots (140k train / 20k validation / 40k held-out test), evaluated on identical held-out shots:

Verdict: MWPM wins decisively — by 5.7× on accuracy and ~98× on per-shot latency. The intervals do not overlap, so this is a resolved result rather than a sampling artefact.

That is the expected and correct outcome, and it is worth being precise about why. The neural decoder is not failing: it cuts the error rate from 22.8% to 8.2% purely from syndrome data, so it has genuinely learned to decode. It loses because pymatching is handed the exact detector error model that generated the shots, while the network has to infer that structure from examples. On simulated noise the baseline is close to optimal, and no amount of training on this data should overturn it.

The regime where learned decoders are expected to win is the one this simulation cannot produce: real hardware, where leakage, crosstalk and drift mean no faithful error model exists to hand the matching decoder.

The latency gap is a separate finding and points at deployment rather than architecture — most of the 4.2 ms is per-call PyTorch dispatch overhead on a batch of one, which is exactly the shape a real feedback loop demands.

MWPM across distance × physical error rate, 40k shots per point:

Below p ≈ 0.006 a bigger code suppresses errors — d=7 is best at every point. Above p ≈ 0.008 the ordering inverts and a bigger code is actively worse, because a larger patch has more places to fail than the extra distance can correct. The crossing is the threshold:

threshold bracketed between p = 0.006 and p = 0.008

This is consistent with the ~0.5–0.7% commonly quoted for the rotated surface code under circuit-level depolarising noise, and it is reported as a bracket because a 7-point sweep does not support a tighter claim.

stim — Craig Gidney's Clifford simulator is what the field actually uses, and it is not a convenience: it samples surface-code detection events at roughly a million shots per second on one core. Decoder benchmarking is statistics-bound, so sampling throughput sets how small an accuracy difference you can resolve. A general-purpose state-vector simulator would make this project impossible.

PyMatching 2 — the standard sparse-blossom MWPM implementation, and the baseline any decoder claim is measured against. Using anything weaker as the baseline would make the neural decoder look good for the wrong reason.

PyTorch (CPU) — the decoder here is small enough that CPU inference is the honest setting: it keeps the latency comparison against MWPM meaningful, since MWPM also runs on CPU. Swapping in a GPU would flatter the neural side of a comparison whose whole point is the real-time budget.

Parquet — syndrome data is wide, boolean and highly compressible, and a threshold sweep produces dozens of parameter combinations. Columnar storage partitioned by experiment slug means the training set for one (d, rounds, p) loads without touching the rest.

src/qecdecoder/
  codes.py       CodeSpec — the single definition of a surface-code experiment
  sampling.py    seeded shot generation → bit-packed Parquet, read back by metadata
  data.py        deterministic train/val/test splits (test touched once)
  model.py       MLPDecoder + NeuralDecoder inference adapter
  train.py       training loop; model selection on validation logical error rate
  baseline.py    MatchingDecoder — the PyMatching MWPM baseline
  metrics.py     Wilson intervals, scoring, per-shot latency measurement
  protocols.py   the Decoder protocol — the extension point
  evaluate.py    paired comparison + threshold sweep and bracketing
  plot.py        dependency-free SVG rendering of a sweep
  cli.py         the qeclab command-line interface
tests/           one module per source module; 90 tests, ~96% line coverage
docs/            threshold.svg (committed); USER_GUIDE.md is workspace-only

uv run pytest -q                       # the full suite
uv run pytest --cov --cov-report=term  # with a coverage report
uv run ruff check . && uv run ruff format --check .

The suite is not only unit coverage. It includes an oracle decoder that cheats by reading the labels — present precisely to prove the harness can detect a genuinely better decoder, so a "baseline wins" result is a finding and not a hard-wired outcome — and a parity-of-distant-detectors task that matching cannot represent by construction, to confirm the neural decoder can learn something matching cannot. CI additionally runs the real pipeline end to end (no stubs) and asserts an event log was written.

This is a simulation harness, not a hardware result.

• Syndromes come from stim's circuit-level depolarising noise model, not from a physical device. Real hardware has leakage, crosstalk, drift and correlated errors that this model does not contain — precisely the regime where a learned decoder is expected to win. Results here therefore understate the case for neural decoding on real devices, and no number in this repo is a prediction about any physical processor.
• The neural decoder is a feed-forward MLP over the full detection-event vector. It is a credible baseline, not a reproduction of AlphaQubit, which uses a recurrent transformer, soft (analogue) readout information, and orders of magnitude more training compute.
• Latency figures are Python-level measurements on a single CPU. A production decoder is FPGA or ASIC and operates under a sub-microsecond budget. The latency comparison here is meaningful relative to the MWPM baseline measured the same way, and meaningless as an absolute real-time claim.
• Threshold estimates come from finite sampling and are reported with confidence intervals. They are not converged to the precision quoted in the literature.

Everything described above runs for real: there are no stubs, no mocked decoders, and no hard-coded results anywhere in the happy path.

• Fowler, Mariantoni, Martinis, Cleland, Surface codes: Towards practical large-scale quantum computation, Phys. Rev. A 86, 032324 (2012) — the threshold and distance behaviour this repo reproduces.
• Gidney, Stim: a fast stabilizer circuit simulator, Quantum 5, 497 (2021) — the simulator used for syndrome generation.
• Higgott & Gidney, Sparse Blossom: correcting a million errors per core second with minimum-weight matching (2023) — the PyMatching 2 algorithm behind the baseline.
• Bausch et al. (Google DeepMind), Learning high-accuracy error decoding for quantum processors (AlphaQubit), Nature (2024) — the learned-decoder result this harness is a scaled-down, honest echo of.

@software{ghosh_qec_decoder_lab,
  author  = {Ghosh, Pralay},
  title   = {qec-decoder-lab: a benchmark harness for neural QEC decoders},
  year    = {2026},
  url     = {https://github.com/ghoshp83/qec-decoder-lab},
  version = {0.2.0}
}

Issues and pull requests are welcome — see CONTRIBUTING.md for the development workflow and the bar a change has to clear. The fastest way to extend the project is to implement the Decoder protocol (Bring your own decoder) and open a PR with the comparison it produces.

MIT — see LICENSE.