Keep your entire Loki stack — Grafana Explore, Drilldown, dashboards, API tooling — and run it on VictoriaLogs.
- Drop-in Loki API. Point your existing Grafana Loki datasource at the proxy. Zero plugin changes, zero query rewrites.
- Measured resource difference. At 310 GiB/day ingest: VL + proxy runs on 1.4 cores and 6.1 GiB RAM. Loki's published minimum for that ingest class: 38 cores, 59 GiB. That gap is real — not a benchmark artifact.
- Proxy intelligence built in. 4-tier cache, 1h window reuse, adaptive parallelism, circuit breaker, rate limits, tenant isolation. One ~14 MB static binary.
Project site: https://reliablyobserve.github.io/Loki-VL-proxy/
Measured head-to-head against tuned Loki: Apple M5 Pro (18 cores, 64 GB RAM), ~8 M log entries across 15 services, 7-day window.
Grafana dashboards auto-refresh every 30 s. After the first fetch, every repeated query is served from in-memory cache without touching VictoriaLogs.
| Workload | Loki P50 | Proxy P50 | Faster by |
|---|---|---|---|
| Small panels (label browser, 1–5 min) | 42 ms | 3 ms | 12× |
| Heavy queries (JSON pipelines, filters) | ~1,800 ms† | 12 ms | 44× |
| Long-range (6 h–72 h, Drilldown) | 4,902 ms | 1 ms | 14× |
| Compute (rate, sum by, quantile, topk) | 4 ms | 4 ms | 10× throughput |
CPU: 6–408× less than Loki. RAM: 1.7–3.9× less for most workloads.
† Loki heavy c=100 was saturated — P90=1,818 ms, P99=6,950 ms.
When many panels hit the same query at once, the proxy collapses them into a single backend call. Everyone gets the result; the backend sees one request instead of N.
| Workload | Loki P50 | Proxy P50 |
|---|---|---|
| Metadata queries | 196 ms | 1 ms |
| Heavy aggregations | 2,399 ms | 1 ms |
| Content search | 13,415 ms | 1 ms |
No cache, no coalescer benefit. Pure translation overhead + HTTP proxying + VL response time.
- Small and metadata queries: at parity with Loki or faster — no penalty
- Heavy queries under load: 1.34× faster at c=10 (179 vs 133 req/s); at c=50 Loki saturates with 35.63% errors while the proxy delivers 1.47× more successful traffic — zero proxy errors at both concurrency levels
- Long-range queries: 2× faster cold — parallel sub-window fetching vs Loki's sequential scan
- Metric aggregations (
rate(),sum by()): 0.4× Loki cold — N VL calls per metric query; historical windows cache after first run (24 h TTL)
Full throughput tables, P90/P99 latency, CPU and RSS breakdowns: Benchmarks · Performance
This is a production deployment, not a synthetic benchmark. The numbers below come from a real VictoriaLogs installation running at 310 GiB/day raw ingest with 800 M total log entries and 7.1 days of retention.
What VictoriaLogs actually consumed at that load:
| Component | Cores | Memory |
|---|---|---|
vlstorage |
1.0 | 5.0 GiB |
vlinsert |
0.1 | 0.6 GiB |
vlselect |
0.1 | 0.25 GiB |
| VL + loki-vl-proxy, combined | ~1.4 | ~6.1 GiB |
For comparison, Loki's own documentation puts the minimum hardware requirement at 38 cores and 59 GiB for the same ingest class (<3 TB/day). That's the floor — a minimal, single-tenant, non-HA deployment.
Caveat: vlselect was measured at zero read concurrency — query load will add to that number. If you run heavy aggregation queries at scale, benchmark your own workload with loki-bench before sizing.
Storage: 2,201 GiB of raw logs (310 GiB/day × 7.1 days) compressed to 40.5 GiB on disk — 54.9× compression. TrueFoundry ran an independent migration and reported ~40% less storage versus their Loki deployment at the same retention.
Replication: Loki's recommended production setup uses RF=3 — tripling write load, disk, and cross-AZ egress. VictoriaLogs is designed for AZ-local deployment with no mandatory replication. If you're paying for cross-AZ data transfer today, that alone can outweigh compute savings.
Migration cost: Zero changes to Grafana, dashboards, alerts, or any Loki API client. The proxy handles translation transparently; remove it and point back at Loki if needed.
Full cost worksheet, scaling projections, and EC2/GCP sizing tables: Cost Model · Scaling
docker run -p 3100:3100 \
ghcr.io/reliablyobserve/loki-vl-proxy:latest \
-backend=http://victorialogs:9428helm install loki-vl-proxy oci://ghcr.io/reliablyobserve/charts/loki-vl-proxy \
--version <release> \
--set extraArgs.backend=http://victorialogs:9428 \
--set extraArgs.patterns-enabled=truePoint your existing Loki datasource at the proxy — no other changes needed.
datasources:
- name: Loki (via VL proxy)
type: loki
access: proxy
url: http://loki-vl-proxy:3100
jsonData:
httpHeaderName1: X-Scope-OrgID
secureJsonData:
httpHeaderValue1: team-alphaThat's it. Grafana Explore, Drilldown, and all dashboards work immediately.
For StatefulSet persistence, peer-cache fleet setup, OTLP push wiring, and image source options, see Getting Started and Operations.
4-tier cache:
- Tier0 — compatibility-edge cache for safe GET responses (no backend hit at all)
- L1 — in-memory hot path
- L2 — disk (bbolt), survives restarts, warms historical windows across large working sets
- L3 — peer cache, lets warm fleet replicas share results instead of all hitting the backend
Window reuse. Long query_range requests are split into 1h windows. Historical windows are served from cache; only the live edge fetches from VictoriaLogs. A 7-day query with warm cache may hit the backend for a single window.
Adaptive parallelism. Parallel window fetches use EWMA-based backpressure — ramps up when VictoriaLogs is fast, backs off automatically before it becomes a problem.
Request coalescing. Concurrent identical queries collapse into one upstream request.
- Grafana Explore — log browsing, filtering, live tail
- Grafana Logs Drilldown — patterns, service view, field breakdown
- Dashboards — all LogQL panel types
- Multi-tenant —
X-Scope-OrgIDisolation with per-tenant rate limits - Live tail — native WebSocket tail or synthetic polling fallback
- Rules and alerts — read bridge to vmalert (no write lifecycle)
- LogQL — 100% coverage: stream selectors, filters, parsers, metric queries, range functions, vector operators
- OTel labels — dotted structured metadata exposed correctly in detected fields, underscore-safe in stream labels
- Circuit breaker — opens on backend failure, closes automatically on recovery
- Per-client rate limits — token bucket, configurable per tenant
- Tenant isolation — strict
X-Scope-OrgIDfanout guardrails; no cross-tenant data bleed - TLS / mTLS — configurable on both northbound (client) and southbound (backend) boundaries
- OTLP push — proxy emits its own traces to any OTLP endpoint
- Operator dashboard — packaged Grafana dashboard covering Client → Proxy → VictoriaLogs, cache behavior, fanout, and resource utilization
- Runbook-backed alerts — 13 alert rules, each with a linked runbook
- 100+ Prometheus metrics — all under
loki_vl_proxy_*prefix - Read-only by default —
/pushblocked, delete gated, debug/admin disabled unless explicitly enabled
flowchart LR
A[Clients<br/>Grafana, Loki API tools, MCP/LLM, scripts]
B[Loki-VL-proxy<br/>Loki API compatibility + translation + shaping + cache]
C[Upstream<br/>VictoriaLogs data + vmalert rules/alerts]
A --> B --> C
classDef client fill:#1f2937,stroke:#60a5fa,color:#f3f4f6,stroke-width:2px;
classDef proxy fill:#172554,stroke:#22d3ee,color:#f8fafc,stroke-width:2px;
classDef upstream fill:#052e16,stroke:#34d399,color:#ecfeff,stroke-width:2px;
class A client;
class B proxy;
class C upstream;
flowchart TD
subgraph L1["Clients"]
G["Grafana<br/>Explore / Drilldown / Dashboards"]
M["MCP / LLM / API Tools"]
C["CLI / SDK Consumers"]
end
subgraph L2["Loki Compatibility Layer (Proxy)"]
API["Loki HTTP + WS API<br/>query / query_range / labels / tail / rules / alerts"]
GUARD["Tenant guardrails + auth context + rate limits + policy checks"]
EDGE["Compatibility-edge cache (Tier0)<br/>safe GET response cache"]
end
subgraph L3["Execution Paths"]
Q["Query translation + shaping"]
T["Tail path (native or synthetic)"]
R["Rules / alerts read bridge"]
RESP["Loki-compatible response"]
end
subgraph L4["Cache Tiers"]
L1C["L1 memory cache"]
L2C["L2 disk cache (optional)"]
L3C["L3 peer cache (optional)"]
end
subgraph L5["Upstream Systems"]
VL["VictoriaLogs<br/>log data"]
VMA["vmalert<br/>rules / alerts state"]
VM["VictoriaMetrics (optional)<br/>recording rule outputs"]
end
G --> API
M --> API
C --> API
API --> GUARD
GUARD --> EDGE
EDGE -->|hit| RESP
EDGE -->|miss| Q
GUARD --> T
GUARD --> R
Q --> L1C
L1C -->|miss| L2C
L2C -->|miss| L3C
L3C -->|miss| VL
VL --> Q
Q --> RESP
T --> VL
R --> VMA
VMA -. optional remote write .-> VM
classDef client fill:#1f2937,stroke:#93c5fd,color:#f3f4f6,stroke-width:2px;
classDef api fill:#0f172a,stroke:#22d3ee,color:#f8fafc,stroke-width:2px;
classDef exec fill:#172554,stroke:#818cf8,color:#eef2ff,stroke-width:2px;
classDef cache fill:#3f1d2e,stroke:#f472b6,color:#fdf2f8,stroke-width:2px;
classDef upstream fill:#052e16,stroke:#34d399,color:#ecfdf5,stroke-width:2px;
class G,M,C client;
class API,GUARD,EDGE api;
class Q,T,R,RESP exec;
class L1C,L2C,L3C cache;
class VL,VMA,VM upstream;
Loki-VL-proxy is validated continuously in CI against three separate tracks: Loki API, Grafana Logs Drilldown, and VictoriaLogs integration.
| Profile | Stream labels (/labels) |
Detected fields / metadata | Best for |
|---|---|---|---|
| Loki-conservative | underscore-only | translated underscore aliases | strict Loki UX |
| Mixed (default) | underscore-only | dotted + translated aliases | Grafana + OTel correlation |
| Native-field | underscore-only (label-style=underscores) |
dotted-native only | VL/OTel-native field workflows |
Grafana query builder works best with underscore aliases. Code mode (label-style=underscores, metadata-field-mode=translated) handles dotted keys without UI tokenization issues.
Tuple safety: Default responses return strict [timestamp, line] 2-tuples. 3-tuple metadata mode activates only when the client sends X-Loki-Response-Encoding-Flags: categorize-labels. Cache keys are segregated by tuple mode.
Stream selectors, filters, parser pipelines, metric queries, range functions, scalar bool comparisons, vector set operators, and invalid LogQL error forms are all covered and machine-validated in CI against a real Loki oracle.
For full detail: Loki Compatibility, Translation Reference, Known Issues
- 100+ Prometheus metrics under
loki_vl_proxy_*— cache hit ratios, window fetch latency, fanout behavior, per-tenant and per-client pressure, circuit breaker state - Packaged operator dashboard — rows for Client-Side Loki API Visibility, Proxy Internal, and Backend-Side VictoriaLogs fanout; fast incident attribution
- 13 runbook-backed alert rules — backend latency, backend unreachable, circuit breaker open, high error rate, rate limiting, tenant isolation, and more
- Structured JSON logs — route-aware, semconv-aligned, with user-pattern attribution from trusted Grafana headers
- OTLP tracing — proxy emits traces to any OTLP endpoint
See Observability and Alert Runbooks Index.
- Read-only API surface by default:
/pushblocked, delete gated, debug/admin disabled - Non-root runtime image, read-only root filesystem, restricted Helm security contexts
- Hardening headers on all HTTP responses including 404s and disabled routes
- CI security gates:
gitleaks,gosec, Trivy,actionlint,hadolint, OpenSSF Scorecard, OWASP ZAP, curated Nuclei - Proxy-specific coverage: tenant isolation, cache boundary enforcement, browser-origin checks on
/tail, forwarded auth handling
See Security and Security Policy.
VictoriaLogs backend with Loki-VL-proxy as the Loki-compatible query layer.
- Getting Started
- Configuration
- Operations
- Architecture
- API Reference
- Security
- Observability
- Performance
- Scaling
- Compatibility Matrix
- Loki Compatibility
- Logs Drilldown Compatibility
- Grafana Loki Datasource Compatibility
- VictoriaLogs Compatibility
- Translation Modes Guide
- Translation Reference
- Alert Runbooks Index
- Deployment Best Practices
- Backend High Latency
- Backend Unreachable
- Circuit Breaker Open
- Client Bad Request Burst
- Proxy Down
- Grafana Tuple Contract
- High Error Rate
- High Latency
- Rate Limiting
- Operational Resources
- Tenant High Error Rate
Apache License 2.0. See LICENSE.