docs: consolidate benchmarks into one file, refresh larger-than-RAM numbers#118
Merged
martinus merged 1 commit intoJul 24, 2026
mastermartinus/oans:masterfrom
docs/consolidate-benchmarksmartinus/oans:docs/consolidate-benchmarksCopy head branch name to clipboard
Merged
docs: consolidate benchmarks into one file, refresh larger-than-RAM numbers#118martinus merged 1 commit intomastermartinus/oans:masterfrom docs/consolidate-benchmarksmartinus/oans:docs/consolidate-benchmarksCopy head branch name to clipboard
martinus merged 1 commit into
mastermartinus/oans:masterfrom
docs/consolidate-benchmarksmartinus/oans:docs/consolidate-benchmarksCopy head branch name to clipboard
Conversation
…umbers Merge docs/benchmarks-low-memory.md into docs/benchmarks.md so all benchmark information lives in a single file with two scenarios: warm re-runs on a large tree (unchanged) and the larger-than-RAM first-run dedupe. Re-ran the larger-than-RAM benchmark (two non-reflinked copies of a Linux kernel tree = 189,546 files / ~10.5 GiB, page cache capped to 4 GiB, cold, 10 interleaved rounds) on merged master (v1.3.0-26-gc20c, now with the Stage 1+2 dedupe rewrite) vs upstream duperemove v0.15.2-2-g897a222: hash+dedupe median oans 13.8 s vs duperemove 179.7 s (~13x) peak RSS (dedupe) oans 121 MiB vs 243 MiB (~2x; streaming pool) hashfile oans 39.7 MiB vs 70.9 MiB (~1.8x) Both tools reach byte-identical sharing (btrfs fi du -s: Exclusive 0 B, Set-shared 4.75 GiB). Repointed the five README links and refreshed the headline numbers (~11x/14 s vs 155 s -> ~13x/13.8 s vs 179.7 s). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The user asked for a single markdown file holding all benchmark information, and for the larger-than-RAM ("2× Linux") dedupe benchmark to be re-run now that the Stage 1+2 dedupe rewrite has landed.
docs/benchmarks-low-memory.mdintodocs/benchmarks.md. The doc now has two clearly-separated scenarios: warm re-runs on a large tree (unchanged) and larger-than-RAM dedupe (refreshed).benchmarks-low-memory.md→benchmarks.md#larger-than-ram-dedupe) and refreshed the README headline figures.Fresh measurement
Two non-reflinked copies of a Linux kernel git tree (189,546 files / ~10.5 GiB), page cache capped to 4 GiB (
MemoryMax=4G MemorySwapMax=0), cold (drop_caches), 10 interleaved rounds.v1.3.0-26-gc20c(master, byte-weighted progress + streaming dedupe pool)v0.15.2-2-g897a222(fork base = upstream 0.15.2)-dr)-r, in-memory)Both tools reach byte-identical sharing — after each run
btrfs filesystem du -s treereportsTotal 10.49 GiB / Exclusive 0.00 B / Set-shared 4.75 GiB. oans reportsReclaimed 5.1 GiB across 45626 groups.Notable vs the previous doc: the dedupe win grew (~11× → ~13×) and oans's dedupe-phase RSS dropped (162 → 121 MiB) thanks to the streaming pipeline — it no longer scales with duplicate groups in flight.
Docs-only; no code changes.
🤖 Generated with Claude Code