Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

fix(coderd/database): order the chat prompt query and its boundary by id - #27619

#27619
Open
ibetitsmike wants to merge 3 commits into
maincoder/coder:mainfrom
mike/chat-message-order/prompt-querycoder/coder:mike/chat-message-order/prompt-queryCopy head branch name to clipboard
Open

fix(coderd/database): order the chat prompt query and its boundary by id#27619
ibetitsmike wants to merge 3 commits into
maincoder/coder:mainfrom
mike/chat-message-order/prompt-querycoder/coder:mike/chat-message-order/prompt-queryCopy head branch name to clipboard

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Stack context

Follows #27495 (merged), which gives chat_messages.id an append-order guarantee and moves the history reads onto it. This PR applies the same fix to the query that builds the model prompt.

Why?

GetChatMessagesForPromptByChatID mixed two orderings. It selected the compaction boundary with created_at DESC, id DESC, then applied that boundary with an id > comparison, and returned rows with created_at ASC, id ASC.

created_at is now(), so it is the transaction start time. Every row in one insert batch shares it, and concurrent transactions can commit in the opposite order to the one they started in. Two consequences, both reaching the provider:

  • Malformed prompts. A tool result could be ordered ahead of the assistant message that requested it. chatprompt.injectMissingToolResults does not repair this: it only handles tool rows already contiguous after an assistant row, and adds missing results. It never moves a tool row that precedes its assistant, and nothing re-sorts the rows in Go.
  • Wrong compaction boundary. The boundary is picked by timestamp but compared by id, so a stale compressed summary could be retained while the actual latest one was dropped.

Changes

Both the boundary CTE and the outer query order by id. The id > predicate is unchanged, which is the point: the ordering now matches the comparison that was always being made.

The boundary index was dead, so it is rebuilt to match. idx_chat_messages_compressed_summary_boundary was created for exactly this lookup, but its predicate requires role = 'system' while compaction writes its summary with the user role (message_conversion.go:334, the only writer of compressed = true). It matched zero rows, and no other query can use it. Migration 000560 rebuilds it as (chat_id, id DESC) WHERE compressed AND NOT deleted AND visibility = 'model', which also matches the new order key.

Measured on PostgreSQL 13 with a 20k-message chat, 11 summaries, and 14 sibling chats so chat_id is selective:

boundary lookup plan buffers
old predicate Index Scan idx_chat_messages_chat, 19,989 rows filtered 267
rebuilt index Index Only Scan 2

Not in scope: the outer SELECT still inspects every row of the chat, because its role = 'system' AND compressed = FALSE disjunct has no lower id bound. That predates this PR and needs a query rewrite rather than an index.

Testing

Two subtests, both verified red by reverting the ORDER BY and regenerating:

  • OrdersByIDWhenTimestampsDisagree returned [4,3,2,1] instead of [1,2,3,4], placing the tool result before the assistant call.
  • CompactionBoundaryUsesID selected the stale summary and leaked the messages between the two summaries into the prompt.

Existing subtests pass unchanged. Migration up/down tests pass, and the rebuilt index was verified red-green: restoring the old predicate returns the plan to a 267-buffer scan, and the old predicate matches 0 rows in the fixture.

Opened by Mux on behalf of Mike.

@ibetitsmike
ibetitsmike force-pushed the mike/chat-message-order/prompt-query branch from 8500551 to 56f7161 Compare July 28, 2026 22:03
@ibetitsmike
ibetitsmike force-pushed the mike/chat-message-order-tiebreaker branch from 1e49a5e to 026bb95 Compare July 28, 2026 22:58
@ibetitsmike
ibetitsmike force-pushed the mike/chat-message-order/prompt-query branch from 56f7161 to d2ce637 Compare July 28, 2026 22:58
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: d2ce6377cf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike force-pushed the mike/chat-message-order-tiebreaker branch from 026bb95 to 5d4893f Compare July 29, 2026 07:05
@ibetitsmike
ibetitsmike force-pushed the mike/chat-message-order/prompt-query branch from d2ce637 to 3ee593d Compare July 29, 2026 07:05
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

Restacked onto the rebased parent, which moved to current main and renumbered its migration to 000559. No content changes here: git range-diff reports both commits identical.

Mux is working on Mike's behalf.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3ee593d604

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/queries/chats.sql
Base automatically changed from mike/chat-message-order-tiebreaker to main July 29, 2026 07:17
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

The prompt query led with created_at while selecting its compaction boundary
by created_at and then applying that boundary with an id comparison. Because
created_at is the transaction start time and is shared across an insert batch,
the prompt could present a tool result before the assistant message that
requested it, and could retain a stale compressed summary as the boundary.
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 3ee593d604

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike force-pushed the mike/chat-message-order/prompt-query branch from 3ee593d to d974c9e Compare July 29, 2026 07:31
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d974c9eee5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/queries/chats.sql
idx_chat_messages_compressed_summary_boundary required role = 'system',
but compaction writes its summary with the user role, so the index never
matched a row. It also led with created_at, which no longer matches the
boundary lookup now that it orders by id.

Rebuilt as (chat_id, id DESC) WHERE compressed AND NOT deleted AND
visibility = 'model'. The boundary lookup becomes an index-only scan:
2 buffers instead of 267 on a 20k-message chat.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 387640a100

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@johnstcn johnstcn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stamp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.