Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

rknightion/tailscale2otel

Open more actions menu

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

495 Commits
495 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

tailscale2otel — Tailscale → OpenTelemetry & Prometheus exporter

Release CI Container Helm chart Go Reference OpenSSF Scorecard License

A single Go binary that turns your Tailscale tailnet into OpenTelemetry metrics, logs and traces over OTLP — or a Prometheus /metrics endpoint, or both at once. Network flow logs, configuration audit logs, device fleet health, key expiry, ACL risk, and tailscaled per-node metrics, exported to Grafana Cloud or any OTEL backend. Headscale is supported too.

📖 Full documentation: m7kni.io/tailscale2otelGetting started · Installation · Configuration · Metrics catalog

186 metrics + 13 log-event types across 15 collectors
18 Tailscale API endpoints consumed polled, streamed, or webhook-driven
89 shipped alert rules 79 Grafana-managed + 10 Prometheus
5 Grafana dashboards 1 flagship 10-tab + 4 legacy-schema
OTLP push (gRPC/HTTP) and/or a Prometheus pull endpoint

Why this exists

Tailscale — the WireGuard-based mesh VPN — exposes a genuinely rich observability surface: network flow logs, configuration audit logs, a detailed device inventory, users, keys, DNS, ACL policy, device posture. But it has no Prometheus endpoint of its own, and it streams logs only to SIEM/storage sinks like Splunk or S3. The existing Tailscale exporters cover a slice of the device API and stop there.

tailscale2otel covers the whole surface and models it properly: semantic-convention-compliant OTEL telemetry, with cardinality control that makes flow logs survivable on a metrics backend.

Things nothing else does

  • Network flow logs as both metrics and logs. Low-cardinality aggregate counters (tailscale.network.io / .packets / .flows) for dashboards and alerting, plus full-fidelity per-connection records as OTEL logs for drill-down — with a top-N rollup (busiest 500 pairs, rest folded to __other__), opt-in port dimensions, and IANA service-name attribution so dst.port: 443 becomes https. This is the feature that usually makes flow logs unaffordable, and it is the reason this project exists.
  • Configuration audit logs → structured OTEL logs + a curated, security-categorized change counter, so you can alert on high-value tailnet changes without ingesting the whole stream.
  • Central tailscaled node-metrics polling. Scrapes each node's native client-metrics endpoint (:5252) from one place instead of deploying a scraper per node — with automatic target discovery from the devices API (tag include/exclude, online-only, address family). Emits both the raw tailscaled_* series and 8 curated tailscale.node.* metrics with folded low-cardinality attributes.
  • Full API-surface coverage — not just devices. Users, auth keys / OAuth clients / API tokens (with expiry), tailnet settings, DNS, ACL policy (scored for structural risk: wildcards, unrestricted rules, auto-approvers, SSH wildcards), device posture / MDM integrations, Tailscale Services, webhook endpoints, contacts, log-stream delivery health, and OAuth apps.
  • Four ingestion paths into one pipeline — poll the API, receive Tailscale's log stream on a built-in Splunk-HEC-compatible receiver, read Tailscale's flow-log export straight out of an S3-compatible bucket, or take real-time HMAC-verified webhooks. All four feed the same processors.
  • Multi-tailnet / MSP mode — one process observing N tailnets, each with its own credentials, and tailscale.tailnet as a real label on every signal (no target_info join required).
  • PII redaction on by default — 13 opt-out categories covering emails, user IDs, hostnames, IPs, node IDs and free-text detail, applied to metric attributes, log bodies and span attributes.
  • API drift CI. Tailscale's API "may change or break without notice", so a decode-fuzz lane gates every PR and three scheduled lanes diff the live OpenAPI spec, track the upstream client library, and hit the real API read-only. See API drift CI.

Quick start

Docker

docker run --rm \
  -e TS2OTEL_TAILSCALE__TAILNET=example.com \
  -e TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_ID=<client-id> \
  -e TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_SECRET=<client-secret> \
  -e TS2OTEL_OTLP__GRAFANA_CLOUD__INSTANCE_ID=<stack-id> \
  -e TS2OTEL_OTLP__GRAFANA_CLOUD__TOKEN=<token> \
  ghcr.io/rknightion/tailscale2otel:latest

No config file needed — every setting has a TS2OTEL_* environment variable. Mount a YAML file and pass -config /etc/tailscale2otel/config.yaml if you prefer.

Kubernetes (Helm)

helm install tailscale2otel oci://ghcr.io/rknightion/charts/tailscale2otel \
  --set-string config.tailscale.tailnet=example.com \
  --set-string secrets.TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_ID=<client-id> \
  --set-string secrets.TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_SECRET=<client-secret>

See Installation for docker-compose, prebuilt binaries (Linux/macOS/Windows × amd64/arm64), and the full chart values.

Binary

go build -o tailscale2otel ./cmd/tailscale2otel
cp config.example.yaml config.yaml   # then edit; secrets stay in env vars
./tailscale2otel -config config.yaml

./tailscale2otel -version                      # print version and exit
./tailscale2otel -validate -config config.yaml # lint a config without starting

No backend? Run it locally

Set TS2OTEL_OTLP__PROTOCOL=stdout to print metrics and logs to the console.

Where the telemetry goes

  • OTLP push (otlp.protocol: grpc|http) with first-class Grafana Cloud support — set otlp.grafana_cloud.{instance_id,token} and the Basic-auth header is built for you. Full TLS/mTLS knobs. Metrics and logs always; traces are opt-in (tracing.enabled) for the exporter's own self-observability, with exemplars linking API-duration histograms to the originating span.
  • Prometheus pull endpoint (prometheus.enabled, off by default) — GET /metrics on its own dedicated listener (default :2112), served alongside OTLP push, with optional bearer/basic auth and TLS. Use it if you already run Prometheus and don't want an OTLP pipeline.
  • stdout for local debugging.

OTLP→Prometheus naming: query the normalized name. Dots become underscores, monotonic counters gain _total, units suffix (By_bytes, s_seconds), and a unit-1 gauge gains _ratio — so tailscale.network.iotailscale_network_io_bytes_total. The full mapping is in the metrics catalog.

Collectors

Collector Cadence Emits
devices 60s online/last-seen/key-expiry/update gauges, NAT & connectivity quality, per-DERP latency, subnet routes, tailnet lock, fleet hygiene roll-ups. Feeds the enrichment cache
flowlogs 60s aggregated traffic counters + per-connection flow logs
auditlogs 60s audit-event logs + a categorized change counter
users 300s user/role/status counts, per-user device & connection gauges, outstanding invites
keys 300s expiry gauges and counts across auth keys, OAuth clients and API tokens
oauth_apps 300s OAuth-application inventory (alpha API; idles silently where unavailable)
settings 600s tailnet feature-toggle gauges
acl 600s ACL size, change detection (by ETag), structural risk scoring
dns 600s nameserver / search-path / split-zone counts, MagicDNS flag
contacts 600s contact verification status (the email itself is never emitted)
webhooks 600s webhook-endpoint inventory + per-endpoint subscription counts
posture_integrations 600s MDM/EDR integration counts, sync health, matched devices
log_stream 600s Tailscale's own SIEM-sink delivery health + delivery counters
services 600s Tailscale Services (VIP) inventory — counts, ports, opt-in backing hosts
node_metrics 60s (opt-in) scrapes tailscaled /metrics endpoints; see above

Each can be disabled or re-tuned. Under provider: headscale the Tailscale-only collectors auto-disable and a reduced set (devices, users, keys, ACL, node-metrics) runs.

Device enrichment depends on the devices collector — flow/audit IP→name resolution silently degrades to unknown/external without it.

Logs: poll, stream or object store — pick one

flowlogs takes a source of poll (default), stream, objectstore or both; auditlogs takes the first three of those. Pick exactly one method per log type — running two risks double-counting, cross-source de-dup is only a best-effort failsafe, and the exporter WARNs at startup when it sees this.

# Poll: tailscale2otel pulls on a schedule (interval/lag/initial_lookback/max_window apply).
flowlogs: { enabled: true, source: poll, interval: 60s, lag: 120s, initial_lookback: 5m, max_window: 1h }

# Stream: Tailscale pushes to the built-in HEC receiver (the window fields are ignored).
flowlogs: { enabled: true, source: stream, log_mode: per_connection }

# Object store: read the export Tailscale writes to S3. No API quota, and the only practical way
# to backfill a long history. Credentials come from the ambient chain (env, IRSA, instance profile).
flowlogs:
  enabled: true
  source: objectstore
  objectstore: { endpoint: https://s3.eu-west-2.amazonaws.com, region: eu-west-2, bucket: my-flow-logs }

Object-store delivery is at-least-once. With the file checkpoint store, successful object identities and failed-object gaps survive restart; transient failures retry with bounded backoff, while invalid compressed objects are quarantined for operator acknowledgement. A scanner error after partial emission can replay already-emitted rows, and OTLP/backend acknowledgement is outside this boundary.

Checkpoints persist how far poll and object-store collectors have read. Details on all paths, receiver auth, object-gap handling, and auto_configure are in Streaming & webhooks.

Dashboards, alerts & the admin UI

  • Dashboardsdeploy/grafana/ ships a flagship 10-tab dashboard (Overview, Fleet & Devices, Network & Flows, Events & Logs, Security & Audit, Policy & Config, Node Metrics, Tailnets, Exporter Diagnostics, Cardinality & Cost) on Grafana's v2 schema (Grafana 13+), with dynamic rendering so a section only appears when its data is present — plus 4 standalone legacy-schema dashboards for older stacks. See Dashboards.
  • Alertsdeploy/alerts/ ships 79 Grafana-managed rules and 10 Prometheus rules. See Alerts.
  • Admin status page — on by default at :9091. Liveness/readiness probes at /healthz and /readyz (never auth-gated), a live status page at /, and the same snapshot at /api/status.json: per-collector health, active-series cardinality with per-label breakdown, the full metrics/log catalog, discovered node targets, and a redacted config summary. Entirely self-contained — no CDN assets, so it renders on an air-gapped tailnet. Auth fails closed on a non-loopback bind with no admin.auth.token.
  • Continuous profiling is opt-in — pprof on the admin server (for Grafana Alloy to pull), or push to Pyroscope / Grafana Cloud Profiles.

Configuration

Layered, lowest precedence first: built-in defaultsoptional YAML fileenvironment variables. Every field is settable as TS2OTEL_ + the dotted key path with __ between levels:

Config key Environment variable
tailscale.auth.oauth.client_secret TS2OTEL_TAILSCALE__AUTH__OAUTH__CLIENT_SECRET
otlp.endpoint TS2OTEL_OTLP__ENDPOINT
collectors.flowlogs.interval TS2OTEL_COLLECTORS__FLOWLOGS__INTERVAL

An unrecognised TS2OTEL_* variable is logged as a WARN at startup — usually a typo.

Authentication: prefer an OAuth client (auto-refreshing, least-privilege all:read) over an API key. Keyless workload identity (OIDC token exchange, e.g. a Kubernetes projected service-account token) is also supported, and every secret has a *_file variant for Docker/Kubernetes secrets.

Full configuration reference · every TS2OTEL_* variable · config.example.yaml

Documentation

Getting started Zero to first metrics in Grafana Cloud
Installation Docker, Helm, compose, binaries
Configuration Every key, default and gotcha
Metrics catalog All 186 metrics and 13 log events
Node metrics Central tailscaled scraping
Streaming & webhooks HEC receiver and webhooks
Architecture How it fits together
Security Data handling, PII, receiver auth
Troubleshooting When it doesn't work

Development

go build ./... && go vet ./... && go test -race ./...
golangci-lint run

Small single-purpose packages under internal/: telemetry (OTEL facade), collector (scheduler/registry/checkpoints + one package per source), tsapi (Tailscale client), provider/hsapi (control-plane abstraction + Headscale), flowlog/audit (records + processors), enrich (device cache), rdns, config, and the stream/webhook receivers. Four committed files are generated — run scripts/regen-generated.sh before committing changes that touch them.

API drift CI

Tailscale's API and OpenAPI spec evolve continuously ("may change or break without notice"), which has broken decoders here before. Four lanes guard it:

Lane When What it checks
Schema-driven decode fuzz every PR (gates) synthesizes payloads from the vendored OpenAPI spec + known wire quirks (numeric proto, polymorphic audit old/new) through the real decoders
OpenAPI drift weekly diffs the live spec against the vendored copy, scoped to consumed operations, classifying breaking vs informational
Client-lib tracking weekly builds and tests against tailscale-client-go/v2@main and @latest
Live contract weekly hits the real API read-only and asserts every consumed GET still decodes

Scheduled lanes are advisory — they open a deduplicated tracking issue and fail the run, but never block PRs. Only the decode-fuzz lane gates.

Maintainer one-time setup
gh label create api-drift -c FBCA04
gh label create clientlib-drift -c FBCA04
gh label create live-contract -c FBCA04

The live lane stores no long-lived Tailscale key. It runs on a standard GitHub-hosted runner and mints a short-lived token from Tailscale's OAuth endpoint using a read-only (all:read) OAuth client, whose TS_OAUTH_CLIENT_ID / TS_OAUTH_CLIENT_SECRET are repo secrets. Keeping them as secrets is safe because this lane is schedule + workflow_dispatch only, so a fork PR can never run it and never reach them; the minted token is masked and lives only for that run. Set the repo variable TS_TAILNET (the tailnet name is not a secret). Missing configuration fails the lane loudly rather than self-skipping, so a misconfigured preflight cannot look green. Optionally set the ANTHROPIC_API_KEY secret for Claude enrichment on the spec-drift and live lanes; the client-lib lane never receives it by design, since it builds untrusted upstream code.

License

Apache License 2.0 — full text in LICENSE; third-party attribution and bundled notices/SBOMs in LICENSING.md.

About

Tailscale exporter for OpenTelemetry and Prometheus. Device fleet, network flow logs and audit logs as OTLP metrics & logs. Single Go binary, Grafana Cloud ready. Headscale supported.

Topics

Resources

Security policy

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

Morty Proxy This is a proxified and sanitized view of the page, visit original site.