fix(storage): refresh stats after txn retry#20098
fix(storage): refresh stats after txn retry#20098SkyFan2002 wants to merge 1 commit intodatabendlabs:maindatabendlabs/databend:mainfrom SkyFan2002:fix-txn-retry-stats-row-countSkyFan2002/databend:fix-txn-retry-stats-row-countCopy head branch name to clipboard
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6f24d49281
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if !merged.is_empty() { | ||
| latest_metas.hll = Some(encode_column_hll(&merged)?); | ||
| latest_metas.row_count += insert_row; | ||
| latest_metas.row_count += stats_row_delta; |
There was a problem hiding this comment.
Avoid double-counting retried stats refreshes
When a retried transaction refreshes statistics for existing rows (for example ANALYZE TABLE inside an active multi-statement transaction) and another concurrent commit has already refreshed the same base rows, stats_row_delta is still positive even though those rows are already represented in latest_metas. The HLL merge deduplicates the values, but this blind row-count increment does not; e.g. base actual=200/stats=100, txn A analyzes to 200, txn B analyzes and commits to 200, then txn A retries and stores stats_row_count=300. This makes system.statistics.stats_row_count exceed the actual row count until stats are rebuilt again.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Handled by narrowing the retry merge to commit-local inserted stats rows only. The helper now returns 0 for pure stats refreshes where table row_count did not increase, and caps any stats delta by the table row delta so refresh coverage cannot be blindly added to latest_metas.row_count.
6f24d49 to
2451a06
Compare
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
Fix explicit transaction commit retry so snapshot statistics row count is merged from the transaction-generated snapshot metadata instead of the multi-table-insert-only row buffer.
Preserve HLL merge behavior while using the additional stats row delta as the source of truth for retried commits.
Add a stateless regression that reproduces the concurrent commit sequence from the issue and verifies
system.statistics.stats_row_countmatchesactual_row_countafter retry.fixes: Explicit transaction COMMIT retry can leave snapshot statistics row count stale #20007
Tests
Validation run locally:
cargo fmt -p databend-common-storages-fuse --checkgit diff --checkpython3 -m py_compile tests/suites/0_stateless/01_transaction/01_04_txn_snapshot_retry.pycargo test -p databend-common-storages-fuse additional_stats_row_deltaType of change
This change is