Large Language Models (LLMs) fail at simple arithmetic because they treat numbers as tokens and use probabilistic approximation. This repository introduces the Semantic Processing Unit (SPU) — a neurosymbolic architecture that decouples semantic parsing (System 1) from algebraic execution (System 2).
Instead of predicting the answer, the neural network predicts a differentiable matrix operator. The calculation itself is performed by deterministic hardware-accelerated matrix algebra, ensuring 100% accuracy and conservation of invariants even on Out-of-Distribution (OOD) data.
The project is divided into three fundamental proofs of concept:
File: 01_mass_conservation.py
Tests the ability of the network to redistribute resources without losing or creating "matter."
- Baseline (MLP): Fails catastrophically when input values scale by x1000 (OOD).
- SPU: Maintains zero-error mass conservation regardless of scale, thanks to architectural constraints (Softmax-based gating).
File: 02_s3_composition.py
Demonstrates Zero-Shot logic. The network learns only two generators of the group (SwapAB and SwapBC).
- Result: The model successfully derives a new operator (
SwapAC) through matrix multiplication with machine precision, without ever seeing during training.
File: 03_s4_locality.py
Proves that the SPU can isolate variables in complex systems.
- Locality: Operations on elements do not interfere with .
- Commutativity: The model learns that (Disjoint Commutativity).
- Deep Chain: Successfully derives long-distance swaps (e.g., ) through a chain of 5+ matrix multiplications.
The SPU splits the computation graph into four stages:
- Data Path (Nouns): Raw state vector . Never passes through trainable weights.
- Neural Controller (System 1): An MLP that parses the command and generates a raw operator matrix.
- Physics Gate: A Softmax layer that enforces column-stochastic constraints (sum=1), ensuring conservation laws.
- Algebraic Processor (System 2): Hardware MatMul (). Pure math, zero hallucinations.
Full Article on Habr/Medium https://habr.com/ru/articles/1001400/