Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions 5 .changeset/backpressure-scale-up-freeze.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---

Add optional `shouldPauseScaling` to the supervisor consumer pool scaling options to freeze scale-up while it returns true (scale-down stays allowed).
6 changes: 6 additions & 0 deletions 6 .server-changes/supervisor-dequeue-backpressure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
area: supervisor
type: feature
---

Add opt-in dequeue backpressure to the supervisor. When enabled, the supervisor reads a verdict from Redis and pauses dequeuing while the worker cluster is saturated, then resumes once capacity is available. Disabled by default - no behavior change for existing deployments.
2 changes: 2 additions & 0 deletions 2 apps/supervisor/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,15 @@
"@kubernetes/client-node": "^1.0.0",
"@trigger.dev/core": "workspace:*",
"dockerode": "^4.0.6",
"ioredis": "^5.3.2",
"p-limit": "^6.2.0",
"prom-client": "^15.1.0",
"socket.io": "4.7.4",
"std-env": "^3.8.0",
"zod": "3.25.76"
},
"devDependencies": {
"@internal/testcontainers": "workspace:*",
"@types/dockerode": "^3.3.33"
}
}
34 changes: 34 additions & 0 deletions 34 apps/supervisor/src/backpressure/backpressureMetrics.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import { Counter, Gauge, type Registry } from "prom-client";

/** Prometheus metrics for dequeue backpressure. */
export class BackpressureMetrics {
/** 1 while backpressure is engaged (computed signal, set even in dry-run). */
readonly engaged: Gauge<string>;
/** 1 when running in dry-run (gates inert). */
readonly dryRun: Gauge<string>;
/** Dequeue attempts the gate skipped - or would have, in dry-run (labelled). */
readonly skipsTotal: Counter<string>;

constructor(opts: { register: Registry; prefix?: string }) {
const prefix = opts.prefix ?? "supervisor_backpressure";

this.engaged = new Gauge({
name: `${prefix}_engaged`,
help: "1 while dequeue backpressure is engaged (computed signal, regardless of dry-run)",
registers: [opts.register],
});

this.dryRun = new Gauge({
name: `${prefix}_dry_run`,
help: "1 when dequeue backpressure is in dry-run mode (gates inert)",
registers: [opts.register],
});

this.skipsTotal = new Counter({
name: `${prefix}_skipped_dequeues_total`,
help: "Dequeue attempts skipped by backpressure (or would be, in dry-run)",
labelNames: ["dry_run"],
registers: [opts.register],
});
}
}
Loading
Morty Proxy This is a proxified and sanitized view of the page, visit original site.