A self-hosted video archival platform powered by yt-dlp. Index, monitor, and download videos from YouTube and hundreds of other sites — all through a web interface with real-time progress tracking.

• Playlist & Channel Monitoring — Track playlists and channels with configurable strategies: scan from the top (Start), bottom (End), or do a full re-scan (Full). New videos are detected automatically on a cron schedule.
• Concurrent Downloads — Semaphore-controlled download queue with duplicate detection, WebSocket progress updates, and stale process cleanup.
• Video Metadata Storage — Every indexed video's metadata is stored in PostgreSQL, including thumbnails, descriptions, and pruned raw yt-dlp JSON output for future use.
• Flexible Deletion — Granular control over what gets deleted: playlist mappings, database records, and/or physical files on disk. Downloaded orphans are automatically preserved in a "None" bucket.
• Powerful Search — Regex and partial-match search across video titles and URLs, with a global: prefix for cross-playlist searches.
• Signed File URLs — Secure, time-limited download tokens prevent unauthenticated file access.
• Site-Specific Support — Built-in handling for Iwara credentials, browser cookie injection, and HTTP proxy routing through Gluetun VPN.
• Automated Background Jobs — Process cleanup, scheduled playlist updates, and orphan video pruning all run on configurable cron schedules.
• URL Deduplication — Incoming URLs are canonicalized before storage: YouTube video IDs are extracted from any URL form (youtu.be, m.youtube.com, watch?v=&list=...), iwara.tv title slugs are stripped, and tracking parameters (utm_*, si=, pp=) are removed globally. A /dedup endpoint lets you scan and merge any existing duplicate records in one shot.

┌─────────────────────────────────────────────────────────┐
│                     Docker Compose                      │
│                                                         │
│  ┌───────────┐   ┌──────────┐   ┌──────────────────┐   │
│  │  Valkey   │   │ Postgres │   │    pgbackups     │   │
│  │  (Redis)  │   │   (DB)   │   │ (daily backups)  │   │
│  └─────┬─────┘   └────┬─────┘   └────────┬─────────┘   │
│        │              │                   │             │
│        └──────┬───────┘                   │             │
│               │                           │             │
│        ┌──────┴──────┐                    │             │
│        │   yt-diff   │────────────────────┘             │
│        │ (Deno + UI) │                                  │
│        └──────┬──────┘                                  │
│               │                                         │
│        ┌──────┴──────┐                                  │
│        │   Gluetun   │  (optional VPN proxy)            │
│        └─────────────┘                                  │
└─────────────────────────────────────────────────────────┘

1. Clone the repo

   git clone --recurse-submodules https://github.com/sagniKdas53/yt-diff.git
   cd yt-diff

2. Configure environment — edit base.env for shared defaults and the deployment env file for host-specific values:

   # base.env holds shared defaults
   
   # local.env / pi5.env / pi4.env hold deployment-specific values
   HOSTNAME=your.hostname.here
   HOST_SAVE_PATH=/path/to/video/storage
   DB_LOCATION=/path/to/postgres/data
   DB_BACKUP_LOCATION=/path/to/backups

3. Set up secrets — create these files in the secrets/ directory:

   File	Contents
   secrets/db_password.txt	PostgreSQL password
   secrets/secret_key.txt	JWT signing key (any random string)
   secrets/proxy_string.txt	(optional) HTTP proxy URL
   secrets/iwara.json	(optional) {"username": "...", "password": "..."}

4. Generate .env for your deployment

   make local   # or: make pi5 / make pi4

5. Start everything

   docker compose up -d
   
   # Or build locally
   make local && make build && docker compose up -d

6. Verify — curl http://localhost:8888/ytdiff/ping should return pong

7. Register — open the web UI and create your first user account.

ghcr.io/sagnikdas53/yt-diff:master

git clone --recurse-submodules https://github.com/sagniKdas53/yt-diff.git
cd yt-diff

# Warm the backend dependency cache
deno cache --lock=deno.lock index.ts

# Build frontend
cd frontend && npm install && npm run build && cd ..

# Generate deployment env once, then start Postgres + Valkey
make local
docker compose up -d yt-db valkey

# Run the server (pick one)
deno task dev          # basic
deno task cookies      # with cookie auth
deno task proxy        # with proxy
deno task full         # everything enabled

All configuration is done through environment variables. Key settings:

For the full list of 30+ environment variables, see docs/GETTING_STARTED.md.

1. Open the web UI and click the Add button.
2. Paste a YouTube playlist URL, channel URL, or single video URL.
3. Choose a monitoring type:
   • Start — for channels where new uploads appear at the top
   • End — for playlists you append to at the bottom
   • Full — complete re-scan every cycle (bandwidth-intensive)
   • N/A — one-time index, no automatic monitoring
4. The server spawns yt-dlp to index all videos. Progress updates stream to the UI in real-time via WebSocket.

Select videos in the SubList panel and click Download. Files are saved to your configured SAVE_PATH organized by playlist subdirectory. The download queue uses a semaphore to limit concurrent processes and prevent duplicates.

Both the playlist panel and video panel support search with special prefixes:

See docs/search.md for the full search syntax reference.

Three automated cron jobs run in the background:

See docs/AUTOMATED_JOBS.md for detailed behavior.

make local                 # Generate .env from base.env + local.env
make pi5                   # Generate .env from base.env + pi5.env
make pi4                   # Generate .env from base.env + pi4.env
make env TARGET=pi5        # Regenerate .env for a deployment and validate it
make build                 # Build without cache
make check                 # Validate compose config using generated .env
make down                  # Stop all containers
make logs                  # Follow container logs

Copyright © 2026 Sagnik Das

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License v3.0 as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

See LICENSE for the full text, or visit https://www.gnu.org/licenses/gpl-3.0.html.