A single Go binary that turns your Tailscale tailnet into OpenTelemetry metrics, logs and traces
over OTLP — or a Prometheus /metrics endpoint, or both at once. Network flow logs, configuration
audit logs, device fleet health, key expiry, ACL risk, and tailscaled per-node metrics, exported to
Grafana Cloud or any OTEL backend. Headscale is supported too.
📖 Full documentation: m7kni.io/tailscale2otel — Getting started · Installation · Configuration · Metrics catalog
| 186 metrics + 13 log-event types | across 15 collectors |
| 18 Tailscale API endpoints consumed | polled, streamed, or webhook-driven |
| 89 shipped alert rules | 79 Grafana-managed + 10 Prometheus |
| 5 Grafana dashboards | 1 flagship 10-tab + 4 legacy-schema |
| OTLP push (gRPC/HTTP) | and/or a Prometheus pull endpoint |
Tailscale — the WireGuard-based mesh VPN — exposes a genuinely rich observability surface: network flow logs, configuration audit logs, a detailed device inventory, users, keys, DNS, ACL policy, device posture. But it has no Prometheus endpoint of its own, and it streams logs only to SIEM/storage sinks like Splunk or S3. The existing Tailscale exporters cover a slice of the device API and stop there.
tailscale2otel covers the whole surface and models it properly:
semantic-convention-compliant OTEL telemetry, with
cardinality control that makes flow logs survivable on a metrics backend.
- Network flow logs as both metrics and logs. Low-cardinality aggregate counters
(
tailscale.network.io/.packets/.flows) for dashboards and alerting, plus full-fidelity per-connection records as OTEL logs for drill-down — with a top-N rollup (busiest 500 pairs, rest folded to__other__), opt-in port dimensions, and IANA service-name attribution sodst.port: 443becomeshttps. This is the feature that usually makes flow logs unaffordable, and it is the reason this project exists. - Configuration audit logs → structured OTEL logs + a curated, security-categorized change counter, so you can alert on high-value tailnet changes without ingesting the whole stream.
- Central
tailscalednode-metrics polling. Scrapes each node's native client-metrics endpoint (:5252) from one place instead of deploying a scraper per node — with automatic target discovery from the devices API (tag include/exclude, online-only, address family). Emits both the rawtailscaled_*series and 8 curatedtailscale.node.*metrics with folded low-cardinality attributes. - Full API-surface coverage — not just devices. Users, auth keys / OAuth clients / API tokens (with expiry), tailnet settings, DNS, ACL policy (scored for structural risk: wildcards, unrestricted rules, auto-approvers, SSH wildcards), device posture / MDM integrations, Tailscale Services, webhook endpoints, contacts, log-stream delivery health, and OAuth apps.
- Four ingestion paths into one pipeline — poll the API, receive Tailscale's log stream on a built-in Splunk-HEC-compatible receiver, read Tailscale's flow-log export straight out of an S3-compatible bucket, or take real-time HMAC-verified webhooks. All four feed the same processors.
- Multi-tailnet / MSP mode — one process observing N tailnets, each with its own credentials, and
tailscale.tailnetas a real label on every signal (notarget_infojoin required). - PII redaction on by default — 13 opt-out categories covering emails, user IDs, hostnames, IPs, node IDs and free-text detail, applied to metric attributes, log bodies and span attributes.
- API drift CI. Tailscale's API "may change or break without notice", so a decode-fuzz lane gates every PR and three scheduled lanes diff the live OpenAPI spec, track the upstream client library, and hit the real API read-only. See API drift CI.
docker run --rm \
-e TS2OTEL_TAILSCALE__TAILNET=example.com \
-e TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_ID=<client-id> \
-e TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_SECRET=<client-secret> \
-e TS2OTEL_OTLP__GRAFANA_CLOUD__INSTANCE_ID=<stack-id> \
-e TS2OTEL_OTLP__GRAFANA_CLOUD__TOKEN=<token> \
ghcr.io/rknightion/tailscale2otel:latestNo config file needed — every setting has a TS2OTEL_* environment variable. Mount a YAML file and
pass -config /etc/tailscale2otel/config.yaml if you prefer.
helm install tailscale2otel oci://ghcr.io/rknightion/charts/tailscale2otel \
--set-string config.tailscale.tailnet=example.com \
--set-string secrets.TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_ID=<client-id> \
--set-string secrets.TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_SECRET=<client-secret>See Installation for docker-compose, prebuilt binaries (Linux/macOS/Windows × amd64/arm64), and the full chart values.
go build -o tailscale2otel ./cmd/tailscale2otel
cp config.example.yaml config.yaml # then edit; secrets stay in env vars
./tailscale2otel -config config.yaml
./tailscale2otel -version # print version and exit
./tailscale2otel -validate -config config.yaml # lint a config without startingSet TS2OTEL_OTLP__PROTOCOL=stdout to print metrics and logs to the console.
- OTLP push (
otlp.protocol: grpc|http) with first-class Grafana Cloud support — setotlp.grafana_cloud.{instance_id,token}and the Basic-auth header is built for you. Full TLS/mTLS knobs. Metrics and logs always; traces are opt-in (tracing.enabled) for the exporter's own self-observability, with exemplars linking API-duration histograms to the originating span. - Prometheus pull endpoint (
prometheus.enabled, off by default) —GET /metricson its own dedicated listener (default:2112), served alongside OTLP push, with optional bearer/basic auth and TLS. Use it if you already run Prometheus and don't want an OTLP pipeline. - stdout for local debugging.
OTLP→Prometheus naming: query the normalized name. Dots become underscores, monotonic counters gain
_total, units suffix (By→_bytes,s→_seconds), and a unit-1gauge gains_ratio— sotailscale.network.io→tailscale_network_io_bytes_total. The full mapping is in the metrics catalog.
| Collector | Cadence | Emits |
|---|---|---|
devices |
60s | online/last-seen/key-expiry/update gauges, NAT & connectivity quality, per-DERP latency, subnet routes, tailnet lock, fleet hygiene roll-ups. Feeds the enrichment cache |
flowlogs |
60s | aggregated traffic counters + per-connection flow logs |
auditlogs |
60s | audit-event logs + a categorized change counter |
users |
300s | user/role/status counts, per-user device & connection gauges, outstanding invites |
keys |
300s | expiry gauges and counts across auth keys, OAuth clients and API tokens |
oauth_apps |
300s | OAuth-application inventory (alpha API; idles silently where unavailable) |
settings |
600s | tailnet feature-toggle gauges |
acl |
600s | ACL size, change detection (by ETag), structural risk scoring |
dns |
600s | nameserver / search-path / split-zone counts, MagicDNS flag |
contacts |
600s | contact verification status (the email itself is never emitted) |
webhooks |
600s | webhook-endpoint inventory + per-endpoint subscription counts |
posture_integrations |
600s | MDM/EDR integration counts, sync health, matched devices |
log_stream |
600s | Tailscale's own SIEM-sink delivery health + delivery counters |
services |
600s | Tailscale Services (VIP) inventory — counts, ports, opt-in backing hosts |
node_metrics |
60s | (opt-in) scrapes tailscaled /metrics endpoints; see above |
Each can be disabled or re-tuned. Under provider: headscale the Tailscale-only collectors
auto-disable and a reduced set (devices, users, keys, ACL, node-metrics) runs.
Device enrichment depends on the devices collector — flow/audit IP→name resolution silently
degrades to unknown/external without it.
flowlogs takes a source of poll (default), stream, objectstore or both; auditlogs takes
the first three of those. Pick exactly one method per log type — running two risks
double-counting, cross-source de-dup is only a best-effort failsafe, and the exporter WARNs at startup
when it sees this.
# Poll: tailscale2otel pulls on a schedule (interval/lag/initial_lookback/max_window apply).
flowlogs: { enabled: true, source: poll, interval: 60s, lag: 120s, initial_lookback: 5m, max_window: 1h }
# Stream: Tailscale pushes to the built-in HEC receiver (the window fields are ignored).
flowlogs: { enabled: true, source: stream, log_mode: per_connection }
# Object store: read the export Tailscale writes to S3. No API quota, and the only practical way
# to backfill a long history. Credentials come from the ambient chain (env, IRSA, instance profile).
flowlogs:
enabled: true
source: objectstore
objectstore: { endpoint: https://s3.eu-west-2.amazonaws.com, region: eu-west-2, bucket: my-flow-logs }Object-store delivery is at-least-once. With the file checkpoint store, successful object identities and failed-object gaps survive restart; transient failures retry with bounded backoff, while invalid compressed objects are quarantined for operator acknowledgement. A scanner error after partial emission can replay already-emitted rows, and OTLP/backend acknowledgement is outside this boundary.
Checkpoints persist how far poll and object-store collectors have read. Details on all paths,
receiver auth, object-gap handling, and auto_configure are in
Streaming & webhooks.
- Dashboards —
deploy/grafana/ships a flagship 10-tab dashboard (Overview, Fleet & Devices, Network & Flows, Events & Logs, Security & Audit, Policy & Config, Node Metrics, Tailnets, Exporter Diagnostics, Cardinality & Cost) on Grafana's v2 schema (Grafana 13+), with dynamic rendering so a section only appears when its data is present — plus 4 standalone legacy-schema dashboards for older stacks. See Dashboards. - Alerts —
deploy/alerts/ships 79 Grafana-managed rules and 10 Prometheus rules. See Alerts. - Admin status page — on by default at
:9091. Liveness/readiness probes at/healthzand/readyz(never auth-gated), a live status page at/, and the same snapshot at/api/status.json: per-collector health, active-series cardinality with per-label breakdown, the full metrics/log catalog, discovered node targets, and a redacted config summary. Entirely self-contained — no CDN assets, so it renders on an air-gapped tailnet. Auth fails closed on a non-loopback bind with noadmin.auth.token. - Continuous profiling is opt-in — pprof on the admin server (for Grafana Alloy to pull), or push to Pyroscope / Grafana Cloud Profiles.
Layered, lowest precedence first: built-in defaults → optional YAML file → environment
variables. Every field is settable as TS2OTEL_ + the dotted key path with __ between levels:
| Config key | Environment variable |
|---|---|
tailscale.auth.oauth.client_secret |
TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_SECRET |
otlp.endpoint |
TS2OTEL_OTLP__ENDPOINT |
collectors.flowlogs.interval |
TS2OTEL_COLLECTORS__FLOWLOGS__INTERVAL |
An unrecognised TS2OTEL_* variable is logged as a WARN at startup — usually a typo.
Authentication: prefer an OAuth client
(auto-refreshing, least-privilege all:read) over an API key. Keyless workload identity (OIDC
token exchange, e.g. a Kubernetes projected service-account token) is also supported, and every
secret has a *_file variant for Docker/Kubernetes secrets.
→ Full configuration reference ·
every TS2OTEL_* variable ·
config.example.yaml
| Getting started | Zero to first metrics in Grafana Cloud |
| Installation | Docker, Helm, compose, binaries |
| Configuration | Every key, default and gotcha |
| Metrics catalog | All 186 metrics and 13 log events |
| Node metrics | Central tailscaled scraping |
| Streaming & webhooks | HEC receiver and webhooks |
| Architecture | How it fits together |
| Security | Data handling, PII, receiver auth |
| Troubleshooting | When it doesn't work |
go build ./... && go vet ./... && go test -race ./...
golangci-lint runSmall single-purpose packages under internal/: telemetry (OTEL facade), collector
(scheduler/registry/checkpoints + one package per source), tsapi (Tailscale client),
provider/hsapi (control-plane abstraction + Headscale), flowlog/audit (records + processors),
enrich (device cache), rdns, config, and the stream/webhook receivers. Four committed files
are generated — run scripts/regen-generated.sh before committing changes that touch them.
Tailscale's API and OpenAPI spec evolve continuously ("may change or break without notice"), which has broken decoders here before. Four lanes guard it:
| Lane | When | What it checks |
|---|---|---|
| Schema-driven decode fuzz | every PR (gates) | synthesizes payloads from the vendored OpenAPI spec + known wire quirks (numeric proto, polymorphic audit old/new) through the real decoders |
| OpenAPI drift | weekly | diffs the live spec against the vendored copy, scoped to consumed operations, classifying breaking vs informational |
| Client-lib tracking | weekly | builds and tests against tailscale-client-go/v2@main and @latest |
| Live contract | weekly | hits the real API read-only and asserts every consumed GET still decodes |
Scheduled lanes are advisory — they open a deduplicated tracking issue and fail the run, but never block PRs. Only the decode-fuzz lane gates.
Maintainer one-time setup
gh label create api-drift -c FBCA04
gh label create clientlib-drift -c FBCA04
gh label create live-contract -c FBCA04The live lane stores no long-lived Tailscale key. It runs on a standard GitHub-hosted runner and
mints a short-lived token from Tailscale's OAuth endpoint using a read-only (all:read) OAuth client,
whose TS_OAUTH_CLIENT_ID / TS_OAUTH_CLIENT_SECRET are repo secrets. Keeping them as secrets is safe
because this lane is schedule + workflow_dispatch only, so a fork PR can never run it and never
reach them; the minted token is masked and lives only for that run. Set the repo variable TS_TAILNET
(the tailnet name is not a secret). Missing configuration fails the lane loudly rather than
self-skipping, so a misconfigured preflight cannot look green. Optionally set the ANTHROPIC_API_KEY secret
for Claude enrichment on the spec-drift and live lanes; the client-lib lane never receives it by
design, since it builds untrusted upstream code.
Apache License 2.0 — full text in LICENSE; third-party attribution and bundled
notices/SBOMs in LICENSING.md.