ORBIT — Open Retrieval-Based Inference Toolkit

Self-hosted AI infrastructure for private RAG and multi-model applications.

Tutorial  |  Docker Guide  |  Cookbook  |  Docs

Maintained by Remsy Schmilinsky

Teams want AI connected to real business data without sending everything to a SaaS vendor, rewriting applications for every model provider, or maintaining fragile glue code between LLMs, databases, APIs, and files.

ORBIT gives you one OpenAI-compatible gateway for private RAG, model routing, retrieval adapters, conversations, tools, and production controls. Run it on your infrastructure, connect the systems you already use, and choose local or hosted models per workload.

You can build:

• Private RAG over documents, databases, APIs, and internal knowledge sources.
• OpenAI-compatible applications that can switch between local models and hosted providers.
• Agent and MCP tools that expose controlled access to business data and actions.
• Agentic tool-calling loops over external MCP servers — filesystem, GitHub, Slack, SharePoint, Postgres, and more — invoked as a skill during any conversation.
• AI media generation pipelines — images and videos from text prompts — wired into the same adapter and conversation system.

1. Connect Postgres, internal PDFs, and a REST API.
2. Run ORBIT on your own infrastructure.
3. Query those sources through one OpenAI-compatible API.
4. Keep sensitive data under your control.
5. Switch between local and hosted models without changing your app.

git clone https://github.com/schmitech/orbit.git && cd orbit/docker
docker compose up -d

Then test the OpenAI-compatible chat API:

curl -X POST http://localhost:3000/v1/chat \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: default-key' \
  -H 'X-Session-ID: local-test' \
  -d '{
    "messages": [{"role": "user", "content": "Summarize ORBIT in one sentence."}],
    "stream": false
  }'

ORBIT listens on port 3000. The admin panel is available at localhost:3000/admin with the default login admin / admin123.

For GPU acceleration:

docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d

Adapter wiring and sample domains live in config/adapters/ and examples/intent-templates/. See the full Docker Guide for GPU mode, volumes, and configuration.

• Route requests across local and hosted models.
• Use one OpenAI-compatible chat API with existing clients and SDKs.
• Stream responses with failover, moderation hooks, and rate limiting.
• Connect through the web chat, Node SDK, or OpenAI-style clients.

• Retrieve context from databases, APIs, files, web content, and vector stores.
• Use intent-based retrieval for natural-language questions over structured data.
• Fan one prompt across multiple sources with composite adapters.
• Build and debug data-backed assistants with template diagnostics and autocomplete.

• Run private RAG over PDFs, documents, images, manuals, contracts, and knowledge bases.
• Reuse retrieved context through conversation threading and cached datasets.
• Handle multilingual conversations across 100+ languages.
• Keep deployments self-hosted for privacy-sensitive environments.

• Generate images from text prompts using DALL-E, Stability AI, and other providers.
• Generate videos from text prompts using Google Veo 2 — prompts are automatically enriched with motion, camera movement, and lighting detail before generation.
• Generated media is persisted server-side and delivered via stable URLs, keeping large binary payloads off the wire.
• Image and video generation follow the same adapter type system as every other capability — no special-casing required.

• Expose controlled tools through MCP for agent clients.
• Connect outward to external MCP servers and run a multi-step tool-calling loop inside a conversation — the model calls tools, ORBIT executes them, feeds results back, and repeats until a final answer is produced.
• Invoke specialized adapters during a conversation with cross-adapter skills.
• Operate with API keys, RBAC, quotas, audit logs, rate limits, and circuit breakers.
• Add voice assistant support through audio adapters.

• Developers building internal AI apps that need real company data, not isolated chat.
• Teams that need private RAG over documents, databases, APIs, and operational systems.
• Companies avoiding long-term lock-in to a single LLM provider or hosted AI platform.
• Engineers connecting AI to SQL, NoSQL, REST, GraphQL, files, and document stores.
• Builders who want OpenAI-compatible APIs with self-hosted control.

ORBIT is probably more than you need if you only want a thin wrapper around one LLM provider.

ORBIT is not only a model router. It handles the layers that usually become custom infrastructure in production RAG systems: retrieval, tools, adapters, conversations, access control, and operational safeguards.

• Retrieval beyond vector search: use intent templates and adapters for structured databases, APIs, files, web content, and vector stores. Intent SQL RAG
• Data source support: query SQL, MongoDB, Elasticsearch, REST, GraphQL, DuckDB, files, and composite sources through one gateway. Composite adapters
• Local and hosted models: run private workloads on Ollama, llama.cpp, vLLM, or other local providers, while still supporting hosted LLMs where appropriate.
• Production controls included: use API keys, RBAC, quotas, audit logging, moderation, rate limits, and circuit breakers. Rate limiting
• Agent-ready protocol support: expose ORBIT-backed chat, RAG, and adapter tools through MCP for agent clients. MCP / OpenClaw walkthrough
• MCP client — agentic tool calling: ORBIT can also connect outward to external MCP servers (filesystem, GitHub, Slack, SharePoint, Postgres, and more), discover their tools, and run a bounded multi-step tool-calling loop. Exposed as the mcp-agent skill on any adapter. Works with OpenAI, Anthropic, Gemini, xAI, and llama.cpp. MCP agent skill

ORBIT sits between clients, models, and data sources. Clients call the OpenAI-compatible API, ORBIT authenticates and routes the request, adapters retrieve or act on external data, and the selected model generates the response with the retrieved context.

git clone https://github.com/schmitech/orbit.git && cd orbit/docker
docker compose up -d

This starts ORBIT with Ollama and SmolLM2, pulls models automatically, and exposes the API on port 3000. The web admin UI is at /admin on the same host.

Connect orbitchat from your host:

ORBIT_ADAPTER_KEYS='{"simple-chat":"default-key"}' npx orbitchat

docker pull schmitech/orbit:basic
docker run -d --name orbit-basic -p 3000:3000 schmitech/orbit:basic

If Ollama runs on your host, add -e OLLAMA_HOST=host.docker.internal:11434 so the container can reach it. The basic image includes simple-chat only.

Download the current release from GitHub Releases, then install:

curl -LO https://github.com/schmitech/orbit/releases/download/v2.7.1/orbit-2.7.1.tar.gz
tar -xzf orbit-2.7.1.tar.gz && cd orbit-2.7.1
cp env.example .env && ./install/setup.sh
./bin/orbit.sh start && tail -f ./logs/orbit.log

• Step-by-Step Tutorial — Chat with your own data in minutes.
• Cookbook — Recipes for database copilots, private gateways, file RAG, voice assistants, fault tolerance, and MCP agents.
• Adapter Configuration — Configure adapters, models, and routing behavior.
• Server Documentation — API, server setup, and MCP protocol details.
• Docker Guide — Docker Compose, GPU mode, volumes, and configuration.

• More ready-to-run adapter templates for common business systems.
• More MCP recipes for agent platforms and desktop clients.
• Persistent MCP server connections and per-server circuit breakers for the MCP agent skill.
• Human-in-the-loop tool approval for write operations in the MCP agent loop.
• Expanded evaluation, tracing, and observability workflows.
• Admin UI improvements for configuration, diagnostics, and operations.
• Additional deployment templates for private cloud and regulated environments.

Contributions are welcome. Check the issues for good first tasks, or open a new issue to discuss your idea.

ORBIT is Apache 2.0, so you can build commercial products on top of it. If ORBIT helps you build private RAG, agent tools, or AI gateway infrastructure, a star helps others find the project and follow its development.

Apache 2.0 — see LICENSE.