fix(coderd/database): order the chat prompt query and its boundary by id - #27619
#27619fix(coderd/database): order the chat prompt query and its boundary by id#27619ibetitsmike wants to merge 3 commits intomaincoder/coder:mainfrom mike/chat-message-order/prompt-querycoder/coder:mike/chat-message-order/prompt-queryCopy head branch name to clipboard
Conversation
8500551 to
56f7161
Compare
1e49a5e to
026bb95
Compare
56f7161 to
d2ce637
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
026bb95 to
5d4893f
Compare
d2ce637 to
3ee593d
Compare
|
Restacked onto the rebased parent, which moved to current
|
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3ee593d604
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
The prompt query led with created_at while selecting its compaction boundary by created_at and then applying that boundary with an id comparison. Because created_at is the transaction start time and is shared across an insert batch, the prompt could present a tool result before the assistant message that requested it, and could retain a stale compressed summary as the boundary.
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
3ee593d to
d974c9e
Compare
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d974c9eee5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
idx_chat_messages_compressed_summary_boundary required role = 'system', but compaction writes its summary with the user role, so the index never matched a row. It also led with created_at, which no longer matches the boundary lookup now that it orders by id. Rebuilt as (chat_id, id DESC) WHERE compressed AND NOT deleted AND visibility = 'model'. The boundary lookup becomes an index-only scan: 2 buffers instead of 267 on a 20k-message chat.
|
@codex review |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Stack context
Follows #27495 (merged), which gives
chat_messages.idan append-order guarantee and moves the history reads onto it. This PR applies the same fix to the query that builds the model prompt.Why?
GetChatMessagesForPromptByChatIDmixed two orderings. It selected the compaction boundary withcreated_at DESC, id DESC, then applied that boundary with anid >comparison, and returned rows withcreated_at ASC, id ASC.created_atisnow(), so it is the transaction start time. Every row in one insert batch shares it, and concurrent transactions can commit in the opposite order to the one they started in. Two consequences, both reaching the provider:chatprompt.injectMissingToolResultsdoes not repair this: it only handles tool rows already contiguous after an assistant row, and adds missing results. It never moves a tool row that precedes its assistant, and nothing re-sorts the rows in Go.Changes
Both the boundary CTE and the outer query order by
id. Theid >predicate is unchanged, which is the point: the ordering now matches the comparison that was always being made.The boundary index was dead, so it is rebuilt to match.
idx_chat_messages_compressed_summary_boundarywas created for exactly this lookup, but its predicate requiresrole = 'system'while compaction writes its summary with the user role (message_conversion.go:334, the only writer ofcompressed = true). It matched zero rows, and no other query can use it. Migration000560rebuilds it as(chat_id, id DESC) WHERE compressed AND NOT deleted AND visibility = 'model', which also matches the new order key.Measured on PostgreSQL 13 with a 20k-message chat, 11 summaries, and 14 sibling chats so
chat_idis selective:idx_chat_messages_chat, 19,989 rows filteredNot in scope: the outer
SELECTstill inspects every row of the chat, because itsrole = 'system' AND compressed = FALSEdisjunct has no loweridbound. That predates this PR and needs a query rewrite rather than an index.Testing
Two subtests, both verified red by reverting the
ORDER BYand regenerating:OrdersByIDWhenTimestampsDisagreereturned[4,3,2,1]instead of[1,2,3,4], placing the tool result before the assistant call.CompactionBoundaryUsesIDselected the stale summary and leaked the messages between the two summaries into the prompt.Existing subtests pass unchanged. Migration up/down tests pass, and the rebuilt index was verified red-green: restoring the old predicate returns the plan to a 267-buffer scan, and the old predicate matches 0 rows in the fixture.