Dub any video into another language — and keep it in sync.
Voxa transcribes a video, translates it with context, speaks it in the target language, and mixes the result back over the original. The engine is one Python file with four speech engines, five translation backends, and one non-negotiable property: the dub does not drift away from the speaker.
pipx install voxa-dub
voxa talk.mp4 --target_lang ru # → talk_dubbed_ru.mp4🎧 Hear the demo dubs — English → Turkish, Azerbaijani and French, produced with no API key
Most dubbing tools concatenate synthesized clips one after another. A clip that runs slightly long pushes the next one later, the error accumulates, and three minutes in the dub is a sentence behind the speaker.
Voxa places every clip anchored to the source timeline. The slot for a line runs from its own onset to the next line's onset. A clip that over-runs is trimmed to fit; a short one is padded with silence. The cursor is assigned, never accumulated — so drift is structurally impossible.
Everything else follows from that decision: the translator is given a character budget so the line fits its slot at a natural pace, speech is only ever sped up (never slowed down, which sounds dragged), and segment starts are tightened to the first word's timestamp so the dub does not begin before the speaker does.
voxa talk.mp4 --target_lang ru
# → talk_dubbed_ru.mp4 + subtitles_ru.srtNo API key is required. The defaults work out of the box.
| Capability | Detail |
|---|---|
| Video dubbing | Dubbed voice mixed over the original as a faint ambience bed; video stream copied without re-encoding |
| Speech recognition | openai-whisper (tiny → turbo) or faster-whisper (2–4× faster, built-in VAD, no torch) |
| Automatic translation | Google, Ollama (local), OpenAI, Anthropic, OpenRouter |
| Context-aware translation | Lines are translated in blocks, not one by one, so pronouns, gender, names and tone stay consistent across a scene |
| Duration-matched translation | Each line gets a character budget, so the dub fits its slot at a natural pace instead of being sped up |
| Preview a run | --dry-run prints the engines, models, output path and which cached steps it would reuse — then exits without writing anything |
| Subtitle processing | SRT output, --subtitles-only mode, built-in SubRip parser (no GPL dependency) |
| Voice cloning | XTTS v2 from a short reference sample, auto-extracted from the source if you don't supply one |
| OpenAI TTS | gpt-4o-mini-tts with instructable delivery |
| Expressive delivery | --detect-emotion selects native voice styles on Edge; on OpenAI TTS an LLM tags each line with an emotion/energy/pace instruction |
| Self-hosted speech | --openai-tts-base-url drives any OpenAI-compatible /v1/audio/speech server — no API key, no extra dependency |
| Offline speech | Piper, fully offline after the model download |
| Anchored placement | Over-runs are trimmed with a fade; short clips are padded; drift cannot accumulate |
| Quality gate | Each clip is transcribed back and scored (word error rate, clipping, silence, pacing), with a per-job report |
| Auto-regeneration | XTTS sampling is stochastic, so a flagged segment is re-synthesized up to twice and the best take is kept |
| Batch processing | Pass several videos in one command |
| Resumable runs | Every step is checkpointed; an interrupted job resumes instead of restarting |
| Preflight validation | Missing input files and a missing FFmpeg are reported before any work begins |
| Configuration | .env loading, JSON config defaults, structured JSON logging |
| Extensible | New speech or translation providers are one adapter plus one registry line |
Note
Voxa ships no model weights and vendors no third-party code. It orchestrates tools you install yourself. See NOTICE.md before using it commercially.
| Layer | Coverage |
|---|---|
| Transcription | Whisper's full language set, auto-detected |
| Translation | 74 target languages carry a full language name into the LLM prompt; Google accepts more |
| Edge TTS | 100+ locales, including native az-AZ voices |
| OpenAI TTS | Multilingual; strongest on major languages |
| Piper | 15 bundled voice mappings, fully offline |
| XTTS | 17 — ar cs de en es fr hi hu it ja ko nl pl pt ru tr zh |
There is no universal answer, and the cloud engine does not always win — a voice that genuinely covers a language usually beats a larger model that only approximates it. Which is why Voxa ships the measurement rather than a verdict.
--quality-gate transcribes every synthesized clip back with a second ASR model and scores it
by word error rate, so you can check the engines against your content:
voxa clip.mp4 --target_lang az --tts edge --quality-gate --gate-model basedocs/BENCHMARK.md has a reproducible script, a worked run across six
languages, and — just as important — the limits of what a number like this can tell you.
Tip
If a language has a dedicated native neural voice in Edge, try it first. For a low-resource
language use --gate-model base or larger: a tiny gate model misreads a perfectly good
voice and reports a worse score than it deserves.
Important
A WER figure only means something next to the clip it came from. The same engine and language score very differently on different source material — one proper noun in a short clip can move the average enormously. Treat any published number, including ours, as a starting point for your own measurement rather than a ranking.
| Stage | Models |
|---|---|
| Transcription | tiny · base · small · medium · large · turbo (large-v3-turbo on the faster backend) |
| Translation (OpenAI) | gpt-5 (default), gpt-5-mini, any chat model id |
| Translation (Anthropic) | claude-opus-4-8 (default), claude-sonnet-5, any model id |
| Translation (Ollama) | llama3 (default), any local model |
| Speech (OpenAI) | gpt-4o-mini-tts |
| Delivery direction | gpt-4o-mini — one cheap call per job, cached |
| Voice cloning | XTTS v2 |
| Offline speech | Piper voices |
| Quality gate | faster-whisper (tiny default, base recommended for low-resource languages) |
Video
│
▼
┌─────────────────────────┐
│ Speech Recognition │ Whisper / faster-whisper
│ │ + word-onset refinement
│ │ + non-speech filtering
└────────────┬────────────┘
▼
┌─────────────────────────┐
│ Translation │ Google · Ollama · OpenAI · Anthropic · OpenRouter
│ │ context-aware, length-budgeted
└────────────┬────────────┘
▼
┌─────────────────────────┐
│ Text-to-Speech │ Edge · OpenAI · Piper · XTTS
│ │ one shared timeline driver
└────────────┬────────────┘
▼
┌─────────────────────────┐
│ Audio Processing │ fit (speed up only)
│ │ anchored placement
│ │ 2-pass loudnorm
└────────────┬────────────┘
▼
Dubbed Video
Every stage is checkpointed in <video>_work/. See docs/ARCHITECTURE.md
for why the cursor is assigned rather than accumulated, how the two provider registries work,
and where each concern lives in the code.
One 15-second English clip, dubbed into three languages from a single transcription. Every dub below was produced with the default engines and no API key:
voxa clip.mp4 --target_lang tr # then --target_lang az, --target_lang fr| Target | Engines | Listen |
|---|---|---|
| 🎬 Original (English) | — | play ▶ · original.mp4 |
| 🇹🇷 Turkish | google + edge |
play ▶ · dub_tr.mp4 |
| 🇦🇿 Azerbaijani | google + edge |
play ▶ · dub_az.mp4 |
| 🇫🇷 French | google + edge |
play ▶ · dub_fr.mp4 |
The second and third languages reused the cached transcription, so only translation and speech were regenerated — the timeline is identical across all three.
1. FFmpeg — Voxa checks for it at startup and tells you if it's missing.
sudo apt install ffmpeg # Debian / Ubuntu
sudo dnf install ffmpeg # Fedora / RHEL
brew install ffmpeg # macOS
winget install Gyan.FFmpeg # Windows2. Voxa — Python 3.9 or newer.
pipx install voxa-dub # recommended: isolated, puts `voxa` on your PATHOr run it once without installing anything:
uvx voxa-dub talk.mp4 --target_lang ruFrom source (for development)
git clone https://github.com/akshinmrv/Voxa
cd Voxa
python3 -m venv venv && source venv/bin/activate # Windows: venv\Scripts\activate
# The CPU-only torch wheel keeps the install considerably smaller.
# Have an NVIDIA GPU? Install the CUDA wheel instead — see "4. GPU acceleration" below.
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install .Note
The distribution is named voxa-dub (the voxa name on PyPI belongs to an unrelated,
abandoned package). The command it installs is voxa — that is what every example below
uses. Running the script directly, python voxa.py …, is equivalent.
Docker — no Python, torch or ffmpeg on the host
docker run --rm -v "$PWD:/data" ghcr.io/akshinmrv/voxa talk.mp4 --target_lang ruThe dub lands next to the input on your host. The defaults need no API key; pass one through
with -e OPENAI_API_KEY=... if you switch engines. Keep the downloaded Whisper models between
runs by mounting the cache:
docker run --rm -v "$PWD:/data" -v voxa-cache:/cache \
ghcr.io/akshinmrv/voxa talk.mp4 --target_lang ruThe image is ~3.7 GB, almost all of it the CPU build of torch. voxa serve runs in the
container too (serve --host 0.0.0.0), but its settings endpoints are restricted to loopback
callers and a port-mapped request does not qualify — run the console natively for those.
3. Optional engines — install only what you use.
| Command | Enables |
|---|---|
pipx install "voxa-dub[faster]" |
--whisper-backend faster — 2–4× quicker, no torch |
pipx install "voxa-dub[piper]" |
--tts piper — fully offline |
pipx install "voxa-dub[anthropic]" |
--translator anthropic |
pipx install "voxa-dub[xtts]" |
--tts xtts voice cloning |
Warning
voxa-dub[xtts] installs coqui-tts, the maintained
community fork. The XTTS-v2 model weights are non-commercial (CPML), and Coqui Inc. no
longer exists to sell a commercial licence. For commercial voice cloning, drive an
MIT-licensed engine through --openai-tts-base-url.
4. GPU acceleration — optional, NVIDIA only.
Voxa detects CUDA on its own; there is no flag to turn it on. What it speeds up:
| Stage | On GPU |
|---|---|
| Whisper transcription (both backends) | ✅ CUDA, fp16 |
| XTTS voice cloning | ✅ CUDA |
| Quality gate (WER scoring) | ✅ CUDA |
| Translation — Google / OpenAI / Anthropic | ➖ network-bound |
| Translation — Ollama | ➖ Ollama uses your GPU on its own |
| Speech — Edge / OpenAI | ➖ cloud |
| Speech — Piper | ➖ CPU (ONNX) |
Install the CUDA build of PyTorch instead of the CPU wheel from step 2:
# Match the CUDA version your driver supports — https://pytorch.org/get-started/locally/
pip install torch --index-url https://download.pytorch.org/whl/cu124Check it took:
python -c "import torch; print(torch.cuda.is_available())" # → TrueWhen the GPU is in use, the run log prints ⚙️ Using device: CUDA at the transcription step
(it prints CPU otherwise). Rough VRAM per Whisper model: tiny/base ≈ 1 GB, small ≈ 2 GB,
medium ≈ 5 GB, turbo ≈ 6 GB, large ≈ 10 GB; XTTS wants roughly 4 GB more.
Note
--whisper-backend faster (CTranslate2) runs on the GPU too, but needs NVIDIA's cuBLAS and
cuDNN libraries: pip install nvidia-cublas-cu12 nvidia-cudnn-cu12.
Voxa checks for CUDA only — Apple Silicon (MPS) and AMD GPUs fall back to CPU.
API keys. Copy .env.example to .env (gitignored) and fill it in. Voxa loads it on
startup; real environment variables always win. Prefer this over --openai_api_key, which
lands in your shell history and the process list.
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
OPENROUTER_API_KEY=sk-or-...Defaults. Put common options in a JSON file. Keys are the long option names with dashes as
underscores; explicit flags override the file. A complete example lives in
examples/config.json.
voxa video.mp4 --config examples/config.jsonLogging. --log-format json emits one JSON object per line. --verbose raises the level
to DEBUG and also surfaces third-party libraries.
All options: voxa --help. It is generated from the code, so unlike a README table it can
never go stale.
# Simplest possible run — no API key, no configuration
voxa video.mp4 --target_lang ru
# Natural, context-aware translation with an LLM
export OPENAI_API_KEY="sk-..."
voxa video.mp4 --target_lang de --translator openai
# Hundreds of models (DeepSeek, Gemini, Llama…) through one OpenRouter key — no extra to install
export OPENROUTER_API_KEY="sk-or-..."
voxa video.mp4 --target_lang az --translator openrouter --openrouter_model deepseek/deepseek-chat
# Clone the original speaker's voice
voxa video.mp4 --target_lang tr --tts xtts
# Fully offline: local LLM translation, offline speech
voxa video.mp4 --target_lang fr --translator ollama --tts piper
# See what a run would do — engines, output path, cached steps — without running it
voxa video.mp4 --target_lang ru --dry-run
# Subtitles only, no synthesis
voxa video.mp4 --target_lang es --subtitles-only
# Several videos in one command
voxa a.mp4 b.mp4 c.mp4 --target_lang ru
# Score the result instead of guessing
voxa video.mp4 --target_lang az --quality-gate --gate-model base
# Self-hosted speech: any OpenAI-compatible server, no API key
voxa video.mp4 --target_lang tr --tts openai \
--openai-tts-base-url http://localhost:8004/v1The CLI is the product. If you'd rather click than type, voxa serve starts a local console
that shells out to the very same voxa command — so the UI cannot drift away from the
engine, because it is the engine. It binds to 127.0.0.1; nothing is uploaded anywhere.
# Backend: REST + SSE, drives the same pipeline per job
pipx install "voxa-dub[serve]"
voxa serve # http://127.0.0.1:8000
# Frontend (separate terminal)
cd web && npm install && npm run dev # http://localhost:3000 → /en/appIt also carries the settings the CLI reads: API keys (kept in your local .env, never echoed
back), per-provider models with a connection test, translation style, and speech style presets.
The repository also contains the public landing site in web/ — a trilingual
(EN/AZ/TR) showcase, deployable as a static site. See web/README.md for
development, environment variables, and deployment.
Voxa/
├── voxa.py # The entire tool: pipeline, engines, registries, CLI
├── voxa_server.py # `voxa serve` operator backend (FastAPI; optional [serve] extra)
├── web/ # Web frontend: public landing + local operator app
├── pyproject.toml # Packaging, extras, ruff and pytest configuration
├── requirements.txt # Core dependencies (optional engines live in extras)
│
├── tests/
│ ├── test_voxa.py # Unit tests: parsing, timing maths, scoring, adapters
│ ├── test_golden.py # Golden harness: the same functions composed
│ └── golden/ # Recorded inputs and expected outputs
│
├── docs/
│ ├── ARCHITECTURE.md # Design decisions and where each concern lives
│ ├── RELEASING.md # Release checklist
│ └── assets/ # Demo videos and images
│
├── examples/
│ └── config.json # A complete, verified configuration file
│
├── .github/
│ ├── workflows/ci.yml # ruff + pytest on Python 3.9–3.12
│ ├── workflows/release.yml # A tag becomes a release only if it passes
│ ├── ISSUE_TEMPLATE/ # Bug report and feature request forms
│ └── dependabot.yml
│
├── README.md
├── CHANGELOG.md
├── CONTRIBUTING.md
├── SECURITY.md
├── CODE_OF_CONDUCT.md
├── NOTICE.md # Third-party licences and commercial-use guidance
└── LICENSE # MIT
Direction, not a promise. Anything requiring hardware or a paid key waits until it can be tested honestly.
| Item | Status | Note |
|---|---|---|
| Parallel synthesis for network engines | Planned | Requests are issued sequentially today; this is the main performance headroom |
| Azure Neural TTS adapter | Blocked | Needs an API key to test. Official az-AZ voices and real SSML prosody |
| Speaker similarity and MOS scoring | Considering | Would extend the quality gate beyond word error rate |
| Wider golden set | Considering | More languages and edge cases in the regression harness |
| Demo assets | Open | Before/after clips in docs/assets/ |
Have a use case that isn't covered? Open an issue.
Do I need an API key?
No. The defaults — Whisper, Google Translate and Edge TTS — need no key. A key is only required for OpenAI/Anthropic translation or OpenAI speech.
Why does the dub not drift out of sync?
Each clip is placed in the slot between its own source onset and the next line's onset, and the timeline cursor is assigned to that next onset rather than accumulated from clip durations. An over-running clip is trimmed instead of pushing everything after it. The timing maths is pure, unit-tested, and locked by a golden regression harness.
Can I use Voxa commercially?
Voxa itself is MIT and requires no copyleft dependency. The engines it drives have their own
licences. --tts piper with --translator ollama is fully permissive; --tts openai with
--translator openai runs under OpenAI's commercial terms. --tts xtts is not commercially
usable. See NOTICE.md.
How do I clone a voice for a commercial product?
Run an MIT-licensed, OpenAI-compatible speech server locally and point Voxa at it:
voxa video.mp4 --tts openai --openai-tts-base-url http://localhost:8004/v1No API key, no extra Python dependency, and no non-commercial model weights.
Can it run entirely offline?
Yes: --translator ollama --tts piper. Nothing leaves your machine after the models are
downloaded.
Does it re-encode my video?
No. The video stream is copied. Only the audio track is rebuilt: the original is mixed in at
5% as a faint ambience bed, the dub at 150%, and amix halves both so the result cannot clip.
Both levels are configurable.
What does a run cost?
Nothing with the defaults. With an LLM translator, token usage is logged after every job; add
your model's prices to the LLM_PRICING table in voxa.py to also print an estimate.
A long job was interrupted. Do I start over?
No. Every step is checkpointed in <video>_work/; rerun the same command and Voxa continues.
Use --no-resume to force a clean start.
Does it work on Windows?
Yes, with FFmpeg on your PATH. CI runs on Linux, and the tool is developed on Windows.
Issues and pull requests are welcome. CONTRIBUTING.md covers the test suite, how to re-record the golden files, and the two dependency rules that matter: no GPL-licensed required dependency, and engine-specific packages belong in extras.
Adding a speech engine is one adapter plus one registry line — the timeline, placement, drift tracking and scoring are already handled for you.
pip install -e ".[dev]"
ruff check .
pytestVoxa is released under the MIT License.
Voxa ships no model weights and vendors no third-party code, but the engines it drives have their own licences — some stricter than Voxa's. Read NOTICE.md before commercial use.
| Configuration | Commercial use |
|---|---|
--tts piper + --translator ollama (fully offline) |
✅ |
--tts openai + --translator openai (paid APIs) |
✅ |
--tts edge / --translator google (defaults, unofficial endpoints) |
|
--tts xtts (XTTS-v2 weights are CPML) |
❌ non-commercial only |
Voxa stands on work done by others.
| Project | Role |
|---|---|
| OpenAI Whisper | Speech recognition |
| faster-whisper | Faster transcription backend |
| edge-tts | Microsoft neural voices |
| Piper | Offline neural speech |
| coqui-tts | Maintained fork powering XTTS voice cloning |
| OpenAI · Anthropic | LLM translation and speech |
| Ollama | Local, private LLM translation |
| FFmpeg | Everything audio and video |
Voxa is built and maintained by Akshin Miranov under the Servoogle name.
Servoogle exists to build practical AI tooling — software that solves a real problem end to end rather than demonstrating a model — and to release the parts that other people can build on. Voxa is one of those parts. Dubbing a video should not require a studio, a licence negotiation, or a proprietary pipeline; it should require one command and a machine you already own.
That is why Voxa is MIT, why it carries no copyleft dependency, why its licence obligations are documented rather than glossed over, and why every engine sits behind a registry that anyone can extend. The interesting problems in this space — keeping a dub in sync, making a synthetic voice sound unhurried, knowing whether the output is actually good — are worth solving in the open.
If Voxa is useful to you, the most valuable thing you can send back is a bug report with the log attached.
Read this in another language: 🇦🇿 Azərbaycan · 🇹🇷 Türkçe
Voxa · MIT · Report a bug · Contribute · Architecture
