O1-Anti is a research architecture that replaces the three most expensive parts of a Transformer, instead of tuning them. Each pillar targets one cost root, and each is validated by a cheap, reproducible go/no-go experiment at toy scale (hundreds of K params, CPU-runnable).

Full design and derivations: O1ANTI_ARCHITECTURE.md.

The four synthetic go/no-go tests (P1–P4) all pass, and the pillars compose in one end-to-end model. The real-text and recall probes are more sobering, and a weight-decay confound runs through them (documented honestly rather than buried):

• E8 (generic LM): NLA trails a matched dense Transformer by ~22% BPB, and four ablations fail to close it. A real, intrinsic limit at this scale.
• E10 (real-text retrieval): RETRACTED. The earlier "NLA beats dense" result was run at the torch-default weight_decay=0.01. Re-running with both models properly regularized (wd=0.1) erases the gap — L=128 becomes a bimodal wash (dense 0.74 vs NLA 0.67), L=512 both fail to grok in budget. NLA's apparent real-text retrieval advantage does not survive a fair comparison.
• E11 (associative recall, MQAR): the one clean positive. At wd=0.1 NLA reliably reaches attention-level recall (1.000 at pairs 8 and 16, multiple seeds, cross-validated) — it matches the ceiling, at a fraction of the cache. It does not exceed attention; the value is parity-quality at lower cache cost. (The Mamba comparison is not yet clean — Mamba was at wd=0.01; needs a GPU re-run.)

Honest current read: the strongest supported claim is NLA matches attention on synthetic content-recall at ~6–8× smaller cache (E11 + the analytic cache win). The real-text retrieval advantage (E10) did not hold up; generic LM (E8) is a real weakness. This is a narrower, efficiency-parity story — not a demonstrated quality advantage on real tasks. Full analysis in O1ANTI_ARCHITECTURE.md § E8/E10/E11; honest positioning in O1ANTI_POSITIONING.md.

Positioning. The honest, evidence-based product thesis — a memory-efficient architecture for retrieval- and long-context-heavy workloads, not a general-purpose Transformer/DeepSeek replacement — and an evidence-gated roadmap are in O1ANTI_POSITIONING.md. Mechanisms proposed but not validated (expert-topology routing — ablated, no benefit; weight-VQ — untested; GWT bottleneck — no measured win) are tracked in DEV_PLAN_REALITY_MAP.md.

Instead of caching K and V (2·d_model per token per layer), each position caches only a compressed state c_j = W_c x_j with d_c ≪ d_model; routing keys and values both derive from c_j, so the inference cache is O(n·d_c). A parallel-scan liquid state s_t summarizes the prefix and gates the output, making the adjacency context-dependent ("liquid"). Each token attends to at most top_k past positions by content score. A straight-through estimator keeps the forward pass sparse while gradients flow through a dense softmax.

No fixed layer stack. A context encoder summarizes the input once; a global router commits to an ordered path of path_len modules out of a library of n_modules. Only modules on the path run — activation ratio is path_len / n_modules. Straight-through Gumbel-softmax routing plus a load-balance penalty prevents module collapse. New capability = grow the library, finetune only the router.

Two non-autoregressive stages: a stage-1 generator produces a compact "semantic skeleton" from the prompt, and a stage-2 parallel decoder cross-attends into it to emit all tokens at once, refined over a fixed number of mask-predict rounds. A length-n generation costs stage1 + decode_iters passes, not n.

Three interchangeable stage-1 skeleton generators (--skeleton_mode), all now at parity: regress (deterministic, 1.000), flow (flow-matching neural ODE, 0.998), discrete (product-quantized VQ codes, 1.000 — fixed from an earlier 0.63 by splitting the codebook into independent subvector groups, see O1ANTI_ARCHITECTURE.md § E9).

The full O1AntiModel conditions generation on the routed module trunk (pillars 1+2) and decodes non-autoregressively (pillar 3), trained end-to-end. It generates at 1.000 accuracy while activating 74% of parameters (the module graph stays sparse — 2 of 6 modules per input) in 7 passes vs 48 for autoregression. The pillars compose without interference.

The decisive fix was a subtle one: position embeddings in the generation stack must use O(1) init, not the usual 0.02. In mask-predict's first round every position is masked, so a tiny position signal makes all queries identical and cross-attention cannot address positions. See O1ANTI_ARCHITECTURE.md § P3.

pip install torch numpy
python -m pytest tests/test_o1anti.py -q                                   # 18 tests

python experiments/p1_nla_swap.py --steps 1500 --seq 32                     # P1
python experiments/p2_module_routing.py --steps 800 --regimes 4            # P2
python experiments/p3_parallel_decode.py --steps 2500 --length 48 \
       --skel_len 48 --decode_iters 6 --skeleton_mode regress             # P3
python experiments/p4_integrated.py --steps 2000 --length 48               # P4

Minimal end-to-end use:

import torch
from o1anti import O1AntiConfig, O1AntiModel

cfg = O1AntiConfig(vocab_size=256, d_model=128, n_modules=8, path_len=4)
model = O1AntiModel(cfg)

ids = torch.randint(0, cfg.vocab_size, (2, 64))
out = model(ids, labels=ids)          # causal-LM path (understanding)
out.loss.backward()

prompt = torch.randint(0, cfg.vocab_size, (2, 16))
tokens = model.generate(prompt, length=48)   # non-autoregressive generation

o1anti/
  config.py        # single O1AntiConfig dataclass
  nla.py           # pillar 1 — NeuralLiquidAdjacency, LiquidStateScan
  module_graph.py  # pillar 2 — ContextEncoder, GlobalRouter, ModuleLibrary
  generation.py    # pillar 3 — SkeletonEncoder/Generator/Prior, VQ, ParallelDecoder
  losses.py        # load balance, state continuity, flow matching
  model.py         # O1AntiModel — LM path + generation path
experiments/       # P1–P4 go/no-go harnesses (p4_integrated = all pillars)
tests/             # 18 tests: shapes, causality, consistency, all modes, P4, multi-head NLA, PQ

O1-Anti's "liquid" ideas descend from MT-LNN (Microtubule-Inspired Liquid Neural Network), the project's first architecture line — a closed-form liquid-time-constant network with a microtubule-inspired protofilament structure that replaces the Transformer FFN. The liquid state-space recurrence in o1anti/nla.py is a direct descendant of MT-LNN's LTC layer, repurposed here to drive sparse routing rather than to replace the FFN.

The biological framing (13 protofilaments, GTP-cap gating, anesthesia-response hooks, IIT-Φ) is inspiration for the priors, not a scientific claim — it motivated the architecture but is not what the benchmarks measure. MT-LNN's own empirical results, papers, pitch decks, and training pipelines remain in this repository (BENCHMARKS.md, mt_lnn_arxiv.pdf, PRD.md, SPEC.md, train.py, train_llama_mt_adapter.py, and the figures/decks). Where any doc disagrees on numbers, the benchmark tables are canonical.

O1-Anti is the forward-looking line: same liquid intuition, but pushed to the architectural level as a systematic answer to Transformer cost, with every claim backed by a reproducible go/no-go experiment.

MIT — see LICENSE.