feat(enterprise): hot-reload aibridged provider pool from DB on pubsub - #24897
#24897Closed
dannykopping wants to merge 1 commit into
graphite-base/24897coder/coder:graphite-base/24897from
dk/aibridge-providers-pool-reloadcoder/coder:dk/aibridge-providers-pool-reloadCopy head branch name to clipboard
Closed
feat(enterprise): hot-reload aibridged provider pool from DB on pubsub#24897dannykopping wants to merge 1 commit intographite-base/24897coder/coder:graphite-base/24897from dk/aibridge-providers-pool-reloadcoder/coder:dk/aibridge-providers-pool-reloadCopy head branch name to clipboard
dannykopping wants to merge 1 commit into
graphite-base/24897coder/coder:graphite-base/24897from
dk/aibridge-providers-pool-reloadcoder/coder:dk/aibridge-providers-pool-reloadCopy head branch name to clipboard
Conversation
This was referenced May 1, 2026
Contributor
Author
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
from
May 1, 2026 18:34
44db962 to
ec333ba
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 1, 2026 18:34
e6263e1 to
c0ca02a
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
from
May 1, 2026 19:24
ec333ba to
87d51ac
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 1, 2026 19:24
c0ca02a to
9d381bb
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
from
May 13, 2026 15:10
87d51ac to
847aded
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 13, 2026 15:19
9d381bb to
ebe85d2
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
from
May 14, 2026 10:57
847aded to
8cb5a08
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 14, 2026 10:57
ebe85d2 to
f50344a
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 14, 2026 11:23
f50344a to
fe2d5e5
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
2 times, most recently
from
May 14, 2026 11:41
eb0f556 to
99f7de6
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 14, 2026 11:41
fe2d5e5 to
42d6c00
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
from
May 14, 2026 13:20
99f7de6 to
9085404
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
2 times, most recently
from
May 14, 2026 13:38
d57f04a to
5d9e052
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
2 times, most recently
from
May 14, 2026 14:13
190628c to
2502c3c
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 14, 2026 14:13
5d9e052 to
59d051d
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
from
May 15, 2026 11:39
2502c3c to
9ac5fc7
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 15, 2026 11:39
59d051d to
0268595
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
from
May 15, 2026 13:16
9ac5fc7 to
d5c5710
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 19, 2026 11:39
c05f349 to
2c8d709
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
from
May 19, 2026 12:52
2f0543c to
3fc3676
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 19, 2026 12:52
2c8d709 to
a229e16
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
from
May 19, 2026 13:03
3fc3676 to
8f98de6
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
3 times, most recently
from
May 20, 2026 08:39
bbe52ac to
f21ec02
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
2 times, most recently
from
May 20, 2026 12:23
d2c57f1 to
be49026
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 20, 2026 12:23
f21ec02 to
771c1ba
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
2 times, most recently
from
May 20, 2026 12:45
d148e00 to
75b4a11
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 20, 2026 12:45
771c1ba to
8c1140e
Compare
dannykopping
changed the base branch from
dk/aibridge-providers-env-migration
to
graphite-base/24897
May 20, 2026 13:15
dannykopping
force-pushed
the
graphite-base/24897
branch
from
May 20, 2026 15:05
75b4a11 to
c3cde94
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 20, 2026 15:05
8c1140e to
0b18288
Compare
dannykopping
changed the base branch from
graphite-base/24897
to
dk/aibridge-providers-env-migration
May 20, 2026 15:06
dannykopping
force-pushed
the
dk/aibridge-providers-env-migration
branch
2 times, most recently
from
May 20, 2026 15:34
7eac2f3 to
d1abba1
Compare
dannykopping
force-pushed
the
dk/aibridge-providers-pool-reload
branch
from
May 20, 2026 15:34
0b18288 to
3d87f3f
Compare
This was referenced May 21, 2026
… on pubsub
Switches the in-memory aibridged daemon from a static, env-derived
provider list to a database-backed list that hot-reloads via pubsub.
After this PR:
- aibridged loads providers from ai_providers at startup (system
actor, dbauthz-gated) and joins them with ai_provider_keys to
pick the operator-preferred primary key (first by created_at).
- Non-Bedrock providers with zero ai_provider_keys are skipped
with a warning; Bedrock providers always have zero keys and
authenticate via the encrypted settings blob (AWS access key +
secret).
- The CRUD handlers from the previous PR publish on
'ai_providers_changed' after every successful Insert/Update/
SoftDelete of a provider AND after every Insert/Delete of a
key, because key changes alone affect the runtime pool.
- Each replica subscribes to that channel and triggers
aibridged.Server.Reload, which atomically swaps the providers
slice on the pool and clears the cached RequestBridge instances.
- In-flight requests continue against their existing
RequestBridge until completion; the cache's OnEvict shutdown
closes MCP connections in the background after a 5-second grace
period.
The proxy daemon is intentionally NOT reloaded yet to keep this PR
focused; it still receives the boot-time provider snapshot. A
follow-up will introduce a Pooler interface for the proxy and mirror
this pattern.
Pool changes:
- CachedBridgePool stores providers via atomic.Pointer[[]Provider]
instead of a fixed slice.
- New Reload(providers) method on the Pooler interface that
atomically swaps the snapshot, calls cache.Clear, and waits for
buffered writes to drain so a subsequent Acquire always sees the
new set.
Tests:
- TestPoolReload covers the happy path: build a pool, acquire a
bridge, Reload, ensure the next Acquire targets the new provider
set.
- TestPoolReloadAfterShutdown ensures Reload is a no-op post-Close
so a stale subscriber notification cannot resurrect a torn-down
pool.
- TestAIProvidersPubsubPublish exercises the producer side: each
of Insert/Update/Delete on a provider emits a notification on
AIBridgeProvidersChangedChannel.
- TestAIProviderKeysPubsubPublish does the same for the keys
sub-resource (Insert and Delete).
This was referenced May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disclaimer: implemented by a Coder Agent using Claude Opus 4.7
Part of the implementation of RFC: Common AI Provider Configs (AIGOV-201).
What this PR does
Switches the in-memory
aibridgeddaemon from a static, env-derived provider list to a database-backed list that hot-reloads via pubsub. After this PR:aibridgedloads providers fromai_providersat startup (system actor, dbauthz-gated).ai_providers_changedafter every successful Insert/Update/SoftDelete on a provider or key.aibridged.Server.Reload, which atomically swaps the providers slice on the pool and clears the cachedRequestBridgeinstances.RequestBridgeuntil completion; the cache'sOnEvictshutdown closes MCP connections in the background after the existing 5-second grace period.Pubsub channel
The channel name (
ai_providers_changed, exported ascoderd.AIProvidersChangedChannel) is provider-generic, not aibridge-specific: any future consumer ofai_providersrows can subscribe. Today aibridged is the only subscriber.Pool changes
CachedBridgePoolstores providers viaatomic.Pointer[[]Provider]instead of a fixed slice.Reload(providers)method on thePoolerinterface that atomically swaps the snapshot, callscache.Clear, andcache.Waits for buffered writes to drain so a subsequentAcquirealways sees the cleared state.Wire-up
enterprise/cli/server.gonow callsloadProvidersFromDB(ctx, db, cfg)instead ofbuildProviders(cfg). The legacy env-drivenbuildProvidersis preserved for the proxy daemon path until the proxy reload follow-up lands.coderd.AIProvidersChangedChannel. Each notification re-loads providers from the database and callsaibridgeDaemon.Reload(...).Tests
TestPoolReload— prime the cache,Reload, observe that the nextAcquireis a fresh build (KeysAdded=1,Misses=1against the post-Reload zeroed metrics).TestPoolReloadAfterShutdown—Reloadis a safe no-op afterShutdown.TestAIProvidersPubsubPublish/TestAIProviderKeysPubsubPublish— end-to-end: each handler mutation publishes a notification on the providers-changed channel.TestAIProvidersCRUD,TestAIProviderKeysCRUD) still pass with the new publish hook in the handlers.Decision log
ai_providers_changedrather thanaibridge_providers_changedbecause the rows it announces are not aibridge-specific. Subscribers are currently aibridged only, but the contract does not assume that.cache.Wait()aftercache.Clear()is required because ristretto's set/clear operations are buffered. Without it, a Reload immediately followed by an Acquire could see the old (cached) bridge.aibridgedown the pubsub subscription, but the package currently has no database/pubsub dependencies. Wiring the subscription inenterprise/cli/server.gokeeps the daemon's interface narrow (Reload(providers)) and matches how other daemon lifecycles are managed.MockPoolerwas regenerated rather than hand-edited so future regen passes are deterministic.