{ "number": 36621, "repo": "bitcoin/bitcoin", "url": "https://github.com/bitcoin/bitcoin/pull/36621", "title": "node: batch block file pruning in a single pass over block index", "author": "mutatrum", "author_association": "CONTRIBUTOR", "created_at": "2026-10-23T11:39:55Z", "updated_at": "2026-10-30T12:50:43Z", "description_hash": "95082b778f355919", "age_days": 9, "draft": false, "labels": [], "milestone": null, "base": "master", "head_sha": "b96e669c607c18c97533f59e70aef44b7c1e613a", "head_ref": "pr-batch-pruning", "head_repo": "mutatrum/bitcoin", "head_history": [ { "t": "2026-10-23T12:40:52Z", "sha": "5786017eee5227365bf9fa28a0a46d83d5cc7164" }, { "t": "2026-10-23T12:54:40Z", "sha": "b96e669c607c18c97533f59e70aef44b7c1e613a" } ], "additions": 115, "deletions": 13, "changed_files": 4, "commit_count": 1, "size_bucket": "M", "mergeable_state": "clean", "bot": { "drahtbot": { "present": true, "reviews": {}, "conflicts": [] } }, "acks_parsed": {}, "acks_tally": { "ack": 0, "stale_ack": 0, "concept_ack": 0, "approach_ack": 0, "nack": 0, "concept_nack": 0, "approach_nack": 0 }, "reviews": { "approved": 0, "changes_requested": 0, "distinct_reviewers": [] }, "signals": { "needs_rebase": false, "ci_failed": false, "mergeable_state": "clean", "last_author_activity": "2026-10-23T12:54:40Z", "last_reviewer_activity": null, "last_reviewer": null, "author_silent_days": 9, "waiting_on_author_days": 0, "days_since_update": 2 }, "refs": { "mentioned": [], "depends_on": [], "fixes": [], "linked_issues": [], "references": [], "conflicts": [] }, "stack": { "shares_commits_with": [], "based_on": [], "base_for": [] }, "review_paths": [], "body": "When starting a node with pruning enabled (e.g. `-prune=550`) on a node with full chain history or pruning many files at once, `FindFilesToPrune` and `FindFilesToPruneManual` previously called `PruneOneBlockFile` once per file.\n\nBecause `PruneOneBlockFile` iterated over the entire `m_block_index` for each file, this was an O(M * N) operation (M = files to prune, N = total blocks). When pruning a full chain (~5,770 files and ~972k blocks), this triggered ~5.6 billion iterations and froze the node running on a Pi 5 for ~10 minutes on startup while holding `cs_main`.\n\nThis PR replaces `PruneOneBlockFile` with `PruneBlockFiles(const std::set&)`, which collects all candidate files and prunes them in a single O(M + N) pass over `m_block_index` using a compact dynamic `bitfield (std::vector)`. Also skips entries without block or undo data to avoid unneeded checks and prevent file 0 pruning from dirtying header-only or already-pruned index entries. The new signature is analogous to `UnlinkPrunedFiles({file_number});`.\n\nBenchmarked with ~972k blocks and 5,770 files to prune:\n- x86_64 (i7-8665U @ 1.90 GHz): 75.1s -> 14.7ms (~5,100x speedup)\n- ARM64 (Raspberry Pi 5): 606.0s -> 26.3ms (>3,500x speedup)\n\nAdd unit test `blockmanager_prune_block_files_batch` to verify batch pruning, empty set no-ops, and clean dirty-set behavior. Update caller tests in `blockmanager_tests` and `rescan_tests`.", "commits": [ { "sha": "b96e669c607c18c97533f59e70aef44b7c1e613a", "date": "2026-10-23T12:54:32Z", "message": "node: batch block file pruning in a single pass over block index\n\nWhen starting the node with pruning enabled (e.g. -prune=550) on a\nnode with full chain history or pruning many files at once,\nFindFilesToPrune and FindFilesToPruneManual previously called\nPruneOneBlockFile once per file.\n\nBecause PruneOneBlockFile iterated over the entire m_block_index for\neach file, this was an O(M * N) operation (M = files to prune,\nN = total blocks). When pruning a full chain (~5,770 files and ~972k\nblocks), this triggered ~5.6 billion iterations and froze the node\nfor ~10 minutes on startup while holding cs_main.\n\nReplace PruneOneBlockFile with PruneBlockFiles(const std::set&),\nwhich collects all candidate files and prunes them in a single O(M + N)\npass over m_block_index using a compact dynamic bitfield\n(std::vector). Also skip entries without block or undo data to\navoid unneeded checks and prevent file 0 pruning from dirtying\nheader-only or already-pruned index entries.\n\nBenchmarked with ~972k blocks and 5,770 files to prune:\n- x86_64 (i7-8665U @ 1.90 GHz): 75.1s -> 14.7ms (~5,100x speedup)\n- ARM64 (Raspberry Pi 5): 606.0s -> 26.3ms (>3,500x speedup)\n\nAdd unit test blockmanager_prune_block_files_batch to verify batch\npruning, empty set no-ops, and clean dirty-set behavior. Update\ncaller tests in blockmanager_tests and rescan_tests." } ], "timeline": [ { "t": "2026-10-23T12:40:52Z", "kind": "force_push", "who": "mutatrum", "commit": "5786017eee5227365bf9fa28a0a46d83d5cc7164", "id": "push:5786017eee5227365bf9fa28a0a46d83d5cc7164" }, { "t": "2026-10-23T12:54:40Z", "kind": "force_push", "who": "mutatrum", "commit": "b96e669c607c18c97533f59e70aef44b7c1e613a", "id": "push:b96e669c607c18c97533f59e70aef44b7c1e613a" } ], "labels_log": [ { "t": "2026-10-23T12:41:37Z", "action": "labeled", "label": "CI failed", "who": "DrahtBot" }, { "t": "2026-10-23T13:43:15Z", "action": "unlabeled", "label": "CI failed", "who": "DrahtBot" } ], "state_log": [], "text_chars": 2830, "text_tokens_estimate": 707, "changed_paths": [ "src/node/blockstorage.cpp", "src/node/blockstorage.h", "src/test/blockmanager_tests.cpp", "src/wallet/test/rescan_tests.cpp" ], "files": [ { "path": "src/node/blockstorage.cpp", "add": 20, "del": 6 }, { "path": "src/node/blockstorage.h", "add": 9, "del": 2 }, { "path": "src/test/blockmanager_tests.cpp", "add": 82, "del": 1 }, { "path": "src/wallet/test/rescan_tests.cpp", "add": 4, "del": 4 } ], "test_lines": 91, "git": { "head": "b96e669c607c18c97533f59e70aef44b7c1e613a", "head_matches_backup": true, "base": "152a55397e451fed9c10041bddf01aa6839847d5", "patch_id": "a067aefe5235a0bd", "commits": [ { "sha": "b96e669c60", "subject": "node: batch block file pruning in a single pass over block index", "files": 4, "add": 115, "del": 13 } ], "patch_truncated": false }, "patch_id": "a067aefe5235a0bd", "input_hash": "1493ea30980c8abe", "extracted_at": "2026-11-02T01:09:49+00:00" }