Binary terrain segmentation in 3,357 parameters — small enough to run bare-metal on a $10 microcontroller. Distilled with quantization awareness · Deployed in pure Rust · 1.7 FPS on an ESP32-S3, no ML accelerator.

Nano-U is a tiny encoder-decoder segmentation network built for autonomous navigation on commodity embedded hardware. Architecture, training, and deployment are co-designed around one hard constraint: 320 KB of Data RAM, no ML accelerator.

Two ideas make it fit:

• Quantization-Aware Distillation (QAD) — a single-pass regime that fuses knowledge distillation from a full-precision BU-Net teacher with INT8 fake-quantization from epoch one.
• A compile-time Rust engine — a fork of MicroFlow that resolves the whole operator graph at build time: no heap allocator, no dynamic dispatch, no interpreter.

⚡ Want more speed? The multicore branch splits inference across both Xtensa cores for ~27% lower latency (~2.35 FPS vs ~1.7 FPS) with bit-identical output. See Branches.

The near-zero Float32→INT8 gap (≈0 pp mIoU on both domains) shows QAD carries accuracy through quantization — the on-device INT8 model matches its Float32 source within measurement noise.

Raw input, ground-truth mask, and Nano-U INT8 prediction across both domains:

The QAD loop: fake-quantization nodes are injected from epoch one, so the optimizer minimizes distillation loss (teacher soft targets) and quantization error (INT8 rounding) at the same time.

Peak RAM measured via stack painting (hardware high-water mark). Power measured at the 5V USB rail, inclusive of LDO, OV2640 camera, and USB-UART bridge.

Nano-U is a strictly sequential 7-stage encoder-decoder. Skip connections are omitted on purpose — holding encoder feature maps in SRAM until the decoder consumes them would blow the 320 KB budget. Depthwise separable convolutions (K=3) are used throughout.

The asymmetric 3×2 MaxPool at Stage 3 reduces 15×20 cleanly to 5×10 with no spatial misalignment. The largest intermediate tensor is 60×80×4 = 19.2 KB (INT8), keeping the arena within budget.

Parameter count is reported at three stages: the plain Float32 architecture has 4,212 weights; QAT wraps these in fake-quant nodes for a saved-checkpoint count of 4,688; and at TFLite export the batch-norm statistics fold into the preceding convolutions, leaving 3,357 stored INT8 parameters with no accuracy loss.

SVD-based redundancy scores (src/nas.py) track layer utilization during training. They empirically confirm that encoder layers generalize across domains while decoder layers are domain-specific — motivating the decoder-only re-distillation strategy in Limitations.

$$\mathcal{L} = \alpha \cdot T^2 \cdot \mathcal{L}_\text{KD} + (1 - \alpha) \cdot \mathcal{L}_\text{CE}$$

where $\mathcal{L}\text{KD}$ is MSE between temperature-scaled sigmoid outputs of teacher and student, $\mathcal{L}\text{CE}$ is binary cross-entropy against hard labels, and $T^2$ compensates for the gradient magnitude reduction from temperature scaling.

Two deployment pipelines live in parallel branches, each paired with a matching MicroFlow branch:

The training/evaluation stack is identical on both — only the firmware inference path differs. See firmware/README.md for the build details.

Nano-U is dataset-agnostic: any binary terrain-segmentation dataset laid out as data/<name>/{train,val,test}/{img,mask} works by pointing config/config.yaml at it. The two below are the ones behind the results in this repo and its accompanying publication.

An outdoor robot navigation benchmark collected in a 48,000 m² unstructured environment. We use 1,181 images from all 5 annotated sequences, split by contiguous sequence (70/20/10) to prevent temporal leakage. Binary traversability masks are derived from the original path class annotations.

Original Source: robot-pesg/BotanicGarden

A custom terrain segmentation dataset captured with the onboard OV2640 camera of an ESP32-CAM mounted on a SunFounder Galaxy RVR rover. 2,659 images across two agricultural environments (tomato and corn fields), annotated with SAM 2. Released alongside this project to support future edge-robotics research.

Requires Python 3.12 (TensorFlow has no wheels for 3.13/3.14).

git clone https://github.com/federico-pizz/Nano-U.git
cd Nano-U
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

The pinned install is CPU-only so it works everywhere; the test suite runs on CPU and GPU-marked tests skip automatically. For an NVIDIA GPU, install a CUDA-capable build:

pip install "tensorflow[and-cuda]==2.20.0"

Very new GPUs (RTX 50-series / Blackwell, sm_120) aren't covered by stock TensorFlow builds yet — they need a CUDA 12.8+ toolkit and a TensorFlow nightly, typically in a separate conda environment. See NVIDIA's Blackwell compatibility guide.

For Rust firmware build and deployment, see firmware/README.md.

INT8 TFLite models for both domains ship in the repo under models/:

models/
├── BotanicGarden/nano_u.tflite
└── TinyAgri/nano_u.tflite

You can evaluate these directly without training from scratch (see Evaluate).

Fill in config/config.yaml with your dataset paths, then pick one of the two (mutually exclusive) options.

Option A — QAD Pipeline (recommended). All four phases in one shot: teacher training → student training with distillation → INT8 TFLite export → evaluation.

python scripts/run_qad.py --config config/config.yaml

Option B — Standard training (no distillation). Trains teacher and student independently.

python scripts/train_model.py bu_net --config config/config.yaml   # Teacher
python scripts/train_model.py nano_u --config config/config.yaml   # Student

python src/evaluate.py nano_u --config config/config.yaml            # held-out test split
python src/evaluate.py nano_u --config config/config.yaml --split val --threshold 0.6

Reports mIoU, Dice, precision/recall and the F0.5/F1/F2 family, plus a precision–recall curve and a metric-vs-threshold sweep. When frames carry sequence ids it adds a per-sequence variance breakdown (plot + *_report.json). Outputs land in results/<dataset>/<model>/. INT8 TFLite and float Keras models go through the same forward path.

Grouped k-fold sweep over distillation temperature/alpha, augmentation regime, the CE-loss ablation, and the conservative Tversky term, with each whole capture sequence kept inside one fold. Selects by mIoU with F0.5 as the safety tiebreak.

python scripts/cv_search.py --config config/config.yaml --k 4 --epochs 200 \
    --temperatures 2 4 8 --alphas 0.3 0.5 0.7 \
    --regimes none geometric photometric full --ce on off \
    --tversky 0.0 0.5 --jobs 3

--ce on off toggles the CE term (off ≡ alpha=1.0); --tversky sweeps the precision-favoring loss (1-w)·BCE + w·Tversky (default 0.0 = pure BCE). --jobs N runs N student pipelines concurrently; the heavy BU-Net teacher has its own --teacher-jobs (default 1). --k must be ≤ the number of distinct sequences.

python scripts/eval_esp32.py nano_u --config config/config.yaml

See firmware/README.md for build setup, binary descriptions, and environment variables.

Nano-U/
├── config/config.yaml          # Configuration template
├── src/
│   ├── models/                 # Nano-U and BU-Net builders
│   ├── utils/                  # Metrics, QAT wrappers, config loader
│   ├── data.py                 # tf.data pipelines and augmentation
│   ├── evaluate.py             # Evaluation and visualization
│   ├── nas.py                  # SVD-based redundancy monitoring
│   ├── quantize_model.py       # INT8 TFLite export and calibration
│   └── train.py                # Training loops (standard + distillation)
├── scripts/
│   ├── run_qad.py              # Full QAD pipeline
│   ├── cv_search.py            # Leakage-safe grouped-CV hyperparameter search
│   ├── train_model.py          # Single-model training
│   ├── eval_esp32.py           # On-device inference and evaluation
│   ├── capture_view.py         # Decode live-camera frame dumps (capture bin) to PNG
│   └── profile_nano_u.py       # Stack painting and energy profiling
├── firmware/                   # ESP32-S3 bare-metal Rust
│   ├── src/bin/
│   │   ├── online.rs                 # Live-camera control loop (OV2640 → nav decision)
│   │   ├── capture.rs                # Live-camera frame dump for pipeline validation
│   │   ├── run.rs                    # Continuous inference loop (default target)
│   │   ├── inference.rs              # One-shot INT8 benchmark over all test images
│   │   ├── single_inference.rs       # Single-image inference + serial output
│   │   ├── analysis.rs               # Stack painting + power profiling (Nano-U)
│   │   └── analysis_person_detect.rs # Stack painting + power profiling (person_detect)
│   ├── src/camera.rs                 # OV2640 live-capture driver
│   ├── src/control.rs                # Pure navigation policy
│   └── build.rs                      # Compile-time quantization param extraction + image packing
├── models/
│   ├── BotanicGarden/nano_u.tflite
│   └── TinyAgri/nano_u.tflite
└── data/
    ├── BotanicGarden/
    └── TinyAgri/

All experiments use an ESP32-S3-CAM (dual-core Xtensa LX7 @ 240 MHz, 512 KB internal SRAM, 16 MB PSRAM, OV2640 camera). No ML accelerator is present; all integer arithmetic runs on the general-purpose ALU. See firmware/README.md for build, binary descriptions, and environment variables.

• Single-core inference (this branch). main runs the operator graph on one Xtensa LX7 core, leaving the second idle. The multicore branch spreads each heavy conv/depthwise layer across both cores for ~27% lower latency with bit-identical output; main stays single-core as the simpler, lower-stack baseline.
• Fixed-domain deployment. Nano-U is re-trained from scratch per domain. SVD redundancy scores suggest the encoder generalizes while the decoder is domain-specific, motivating decoder-only re-distillation for new domains.

If you use Nano-U or the TinyAgri dataset, please cite:

@article{pizzolato2026nanou,
      title={Nano-U: Efficient Terrain Segmentation for Tiny Robot Navigation}, 
      author={Federico Pizzolato and Francesco Pasti and Nicola Bellotto},
      year={2026},
      booktitle={Towards Autonomous Robotic Systems}
}

Dual-licensed under the MIT License and the Apache License 2.0 — use it under either, at your option. See LICENSE for the full text.