#36621 node: batch block file pruning in a single pass over block index

ReviewabilityReviewsAgreementSize
#36621 mutatrum node: batch block file pruning in a single pass over block indexReady0Crickets29 + 86 tests
  • Prevent node freezes when switching an existing node with full history to pruned mode
  • Greatly reduce the time required to prune many block files at once
  • Ready for review
  • No review verdicts recorded
  • No reviews or comments yet
  • Awaiting initial feedback
  • 29 lines added or modified outside tests
  • 86 lines added or modified in tests
  • 13 lines removed in total
  • 4 files, 1 commits
Validation#12 of 53
P2 · speedup
  • P2 because it eliminates long startup hangs holding the main lock when switching a node to pruned mode
  • Reduces pruning time from up to 10 minutes down to milliseconds on low-power hardware like a Raspberry Pi
Ranking notes
Reviewability: ReadyStalePaused
Agreement: StrongPositivePositive w/ caveatsNeutralMildDisputedBlockedCrickets
Reviews: current code-review ACKs, then (+stale ACKs) and -NACKs; greener = more ACKs.
Size: lines added or modified outside tests, then in tests; greener = smaller.

Goal

Replaces `PruneOneBlockFile` with `PruneBlockFiles(const std::set<int>&)` in `BlockManager`, batching block file pruning into a single pass over `m_block_index` using a dynamic bitfield. In `FindFilesToPrune` and `FindFilesToPruneManual`, calls are consolidated so the block index is traversed once across all candidate files rather than once per file. It also skips block index entries that have neither block data nor undo data, avoiding redundant checks and preventing header-only entries from being marked dirty when file 0 is pruned.

Problem: Switching an existing node with full history to pruned mode (or manually pruning many block files) iterated over the entire `m_block_index` separately for every block file pruned. On a full chain (~5,770 files and ~972k blocks), this O(M * N) operation performed billions of iterations while holding `cs_main`, freezing the node for up to ten minutes on startup.

Evidence of importance:

  • Commit description: 'When pruning a full chain (~5,770 files and ~972k blocks), this triggered ~5.6 billion iterations and froze the node running on a Pi 5 for ~10 minutes on startup while holding cs_main.'
  • Author benchmarks: x86_64 (i7-8665U) dropped from 75.1s to 14.7ms (~5,100x speedup); ARM64 (Raspberry Pi 5) dropped from 606.0s to 26.3ms (>3,500x speedup).

Risk of a mistake: medium (validation). A bug in updating block index records or dirty flags during pruning could corrupt block index metadata or incorrectly report block availability.

Changed since the previous assessment: Diff inspected (previously nopatch); implementation confirms the bitfield batching, the dataless/header-only skip, and new unit tests in blockmanager_tests.

Reviewability

Ready

Nothing known that would invalidate a review now.

Agreement

Crickets

no substantive comment either way

Objections, suggestions, and questions: none recorded.

Files

91 lines under test/bench/ci.

  • src/test/blockmanager_tests.cpp +82/-1
  • src/node/blockstorage.cpp +20/-6
  • src/node/blockstorage.h +9/-2
  • src/wallet/test/rescan_tests.cpp +4/-4

Card

The short text the ranking pass compares across PRs.

Batches block file pruning into a single pass over the block index rather than traversing the full index once per pruned file. When starting a node with -prune on an existing full-chain datadir or pruning thousands of files at once, the previous O(M * N) approach executed billions of iterations while holding cs_main, freezing the node for tens of seconds on desktop hardware and up to 10 minutes on a Raspberry Pi 5. Benchmarks report reducing pruning time on 5,770 files from 606s to 26ms on ARM64 and 75s to 15ms on x86_64.

Data

dossier JSON · extract JSON · model ledger, generated 2026-10-25T01:14, confidence high, input hash ef193e9dd0d8a38e
labels: none

Categories

Validation

P2 · speedup

Consolidating block index traversals eliminates an O(M * N) loop holding `cs_main` that froze nodes for up to 10 minutes on startup when switching an existing chain to pruned mode. Author benchmarks show pruning time dropping from 606s to 26ms on a Raspberry Pi 5 and 75s to 15ms on x86 (>3,500x speedup).

Membership: Changes block storage and block index pruning logic in `src/node/blockstorage.cpp` and `src/node/blockstorage.h`.

Factors: security/stability 1, bug 2, performance 3, user value 1, leverage 0