Summary

With an OpenAI Realtime RealtimeModel, syncing the chat context after an interrupted agent turn that has skipped (never-played) outputs throws Unsupported item type: agent_config_update. It is caught and only logged as a warning, but it means the "remove never-played messages" cleanup silently fails.

Versions

• @livekit/agents 1.5.5
• @livekit/agents-plugin-openai 1.5.5
• Node 24, Linux (also reproduces locally)

Observed log / stack

failed to sync chat context to remove never-played messages
  Error: Unsupported item type: agent_config_update
    at livekitItemToOpenAIItem (.../@livekit/agents-plugin-openai/dist/realtime/realtime_model.js:1668)
    at createChatCtxUpdateEvents (.../realtime/realtime_model.js:501)
    at RealtimeSession.updateChatCtx (.../realtime/realtime_model.js:380)
    at AgentActivity (.../@livekit/agents/dist/voice/agent_activity.js:2730)

Root cause

The framework inserts AgentConfigUpdate items into agent._chatCtx (initial one in _startSession at agent_activity.js:385 whenever the agent has instructions/tools; more on instruction/tool changes at 655/684).

The interruption-cleanup path syncs the chat context without excluding those items:

// agent_activity.js ~2727-2730
const anySkipped = messageOutputs.some((output) => output.played === "skipped");
if (anySkipped && realtimeModel.capabilities.midSessionChatCtxUpdate) {
  try {
    await realtimeSession.updateChatCtx(this.agent._chatCtx); // <-- no excludeConfigUpdate
  } catch (error) {
    this.logger.warn({ error }, "failed to sync chat context to remove never-played messages");
  }
}

The plugin's updateChatCtx → createChatCtxUpdateEvents copies the ctx (realtime_model.js:472, no excludeConfigUpdate) and calls livekitItemToOpenAIItem on each to-create item; agent_config_update hits the default: branch and throws (realtime_model.js:1668). AgentConfigUpdate items are never successfully created on the remote side, so they always appear as "to create" and always throw.

Contrast the reuse path a few lines up, which copies correctly:

// agent_activity.js ~488-492 — the correct pattern
this.realtimeSession.chatCtx.copy({ excludeInstructions: true, excludeHandoff: true, excludeConfigUpdate: true })

Suggested fix

Exclude config-update items in the cleanup sync (one line):

await realtimeSession.updateChatCtx(this.agent._chatCtx.copy({ excludeConfigUpdate: true }));

(Alternatively / additionally, handle agent_config_update in livekitItemToOpenAIItem by skipping it.)

Repro

1. OpenAI Realtime RealtimeModel (server-side semantic_vad), AgentSession with turnHandling: { turnDetection: 'realtime_llm' }, agent given instructions + tools.
2. Have the caller interrupt (barge-in) mid agent-speech so an output chunk is skipped, or end the call while the agent is generating.
3. The warning above is logged.

Impact

Caught → non-fatal (the call continues/ends fine), but the never-played-message cleanup is skipped, and it produces error-level noise on every interrupted+skipped turn.