#32427 kernel: Replace leveldb-based BlockTreeDB with WAL and .dat file based store

ReviewabilityReviewsAgreementSize
#32427 sedited kernel: Replace leveldb-based BlockTreeDB with WAL and .dat file based storeWaiting on author0 (+6)Blocked988 + 769 tests
  • Allow external applications using libbitcoinkernel to inspect block and header data concurrently
  • Remove the exclusive database lock that forces users to shut down Bitcoin Core to run indexers
  • Needs rebase
  • Author silent 84 days and reworking parallel reader access after converting to draft
  • Broad interest in removing the LevelDB dependency and enabling parallel indexing (josibake, theuni)
  • Verified by differential fuzzing for 5000 CPU hours without issues (marcofleon)
  • Unaddressed objection: LockDirectory succeeds without locking on re-lock and drops the lock (pzafonte)
  • Unaddressed objection: directory committing is a no-op on Windows risking corruption (stickies-v)
  • 988 lines added or modified outside tests
  • 769 lines added or modified in tests
  • 201 lines removed in total
  • 31 files, 10 commits
Kernel (libbitcoinkernel)#2 of 18
P1 · new feature
  • P1 after comparing with the other PRs here (assessed alone as P2): Raised to P1: LevelDB's exclusive process lock is the concrete reason external consumers (Electrs/Esplora) cannot read block/header data from a running node, which is the category's defining use case.
  • P2 because it allows external consumers to read block and header data while a node is running
  • Unblocks concurrent index building for downstream tools like Electrs and Esplora
Ranking notes
Validation#17 of 53
P2 · new feature
  • P2 because it replaces the block index storage engine, affecting node crash safety and startup migration
  • Directly impacts validation persistence, file locking, and data integrity under power loss
Ranking notes
Reviewability: ReadyStalePaused
Agreement: StrongPositivePositive w/ caveatsNeutralMildDisputedBlockedCrickets
Reviews: current code-review ACKs, then (+stale ACKs) and -NACKs; greener = more ACKs.
Size: lines added or modified outside tests, then in tests; greener = smaller.

Goal

This PR replaces the LevelDB-based block tree database (CBlockTreeDB) with a custom flat-file store (BlockTreeStore) using fixed-size serialized records, a write-ahead log (WAL) for atomic updates, and CRC32C checksums for corruption detection. It also provides automatic migration of existing LevelDB block indexes on startup and introduces directory-level file locking to guard against concurrent writer processes.

Problem: LevelDB acquires an exclusive file lock that prevents any other process from opening the database. External applications building on libbitcoinkernel (such as indexers like Electrs) cannot inspect block or header data concurrently while Bitcoin Core is running, forcing users to shut down the node first.

Reviewability

Paused · Waiting on author

Open objections from pzafonte, ryanofsky, shahsb with no author reply for 31 days; review would repeat what has been asked.

Waiting on (from the thread read):

  • author: author is redesigning the approach for parallel reader access and needs to address LockDirectory re-locking defect 2026-10-01

Agreement

Blocked

objection that needs a change, nobody has replied (pzafonte)

Thread notes: Added pzafonte as a participant with an objection regarding LockDirectory returning Success without locking and destroying either instance releasing the lock.

Objections, suggestions, and questions:

ReviewerTypeHarm, request, or questionStatusMust beAuthor repliedQuote
Sjors (member)suggestion, approachConsider storing one block per file instead of bundling multiple blocks per fileanswered · author declined as out of scope for the block tree store PRansweredyes2025-05-07: 'Have you considered simply having one block per file?'
Status from: 2025-05-07: sedited: 'I think this is a bit out of scope'
ryanofsky (member)objection, correctnessInterrupted in-place writes could corrupt data files without atomic updates or checksum detectionfixed · fix pushed · addressed by adding WAL and checksumschangedyes2025-05-07: 'How worried are we about file corruption here? I thought the main reason we use leveldb and sqlite databases in places like this where we don't need indexing is that they support atomic updates, so yo'
Status from: 2025-06-10: sedited: 'The latest push updates the block tree store to use a write ahead log for atomic writes, and crc32c checksums to detect '
shahsb (none)question, approachHow will concurrent reads/writes be handled, and what are the corruption recovery and migration strategies?openansweredno2025-05-07: 'How will concurrent reads (and potentially writes) be handled with the flat file format?'
ryanofsky (member)suggestion, approachMake BlockTreeStore methods pure virtual with FlatFile and LevelDB subclassesopenansweredno2025-05-08: 'make BlockTreeStore methods pure virtual and have FlatFile and LevelDB subclasses implementing them'
davidgumberg (member)question, safetyCould a miner DoS nodes under a hash-sharded directory format by suffix-mining block hashes?answered · theuni noted hash salting/obfuscation could prevent game-able file distributionansweredby others2025-05-16: 'is there any scenario where a miner could DoS nodes with this format by also mining the last two bytes?'
Status from: 2025-05-16: theuni: 'There's also the possibility of using a local salt like we do for most other game-able data'
willcl-ark (member)objection, correctnessRace between writer creating log_flag.dat and calling ApplyLog allows another process to apply the WAL first, causing the writer to throw an errorfixed · fix pushed · author gated lock file flushing behind read-only flag and introduced OpenModechangedyes2026-06-24: 'Since log_flag.dat is visible before the writer enters the final ApplyLog() critical section, another process can observe the completed WAL, apply it, and clear the flag first.'
Status from: 2026-06-24: sedited: 'Gated lock file flushing behind a read-only flag to avoid the cross-process log file contention issues flagged by @willc'
stickies-v (member)suggestion, safetyAdd a timeout when waiting to acquire StoreLock so the process does not block indefinitelyfixed · fix pushed · author added timeout to lock waiting logicansweredyes2026-06-25: 'Since a goal of this change is to allow other applications to access the database, I think we need to be defensive and assume that a lock can be held without limit. Adding a timeout here seems sensibl'
Status from: 2026-06-29: sedited: 'added a timeout to the lock waiting logic.'
stringintech (contributor)suggestion, approachAdd a separate directory lock for writers only that fails fast on a second writer process instead of busy-waitingfixed · fix pushed · author added write directory lock and functional testansweredyes2026-06-29: 'add a separate directory lock for writers only, one that's taken for the whole lifetime of the store (in WRITE/WIPE mode) and fails fast on a second writer process instead of busy-waiting'
Status from: 2026-06-29: sedited: 'Also introduced a write directory lock that gets taken on instantiation and yielded on destruction.'
willcl-ark (member)question, interfaceWhether not exposing BlockTreeStore::OpenMode::READ in btck_chainstate_manager_create is intentional, and if wiring it through the kernel API is planned as a follow-upanswered · author confirmed intentional and suggested a separate discussionansweredyes2026-06-30: 'looks like btck_chainstate_manager_create() does not currently expose BlockTreeStore::OpenMode::READ, so external kernel users cannot open the block tree store in read-only mode yet. Is that intention'
Status from: 2026-06-30: sedited: 'Yes, intentional. Ideally we'd have a separate discussion on what that would look like.'
stickies-v (member)objection, correctnessDirectory committing is a no-op on Windows, potentially breaking flag file atomicity and leading to inconsistencies on power lossanswered · author converted to draft to address concurrency and storage issueschangedyes2026-07-02: 'This is a no-op on Windows. I think this can lead to several edge case issues, like the block tree being marked as unpruned when blocks have already been removed, or the WAL not being applied atomical'
Status from: 2026-07-09: sedited: 'Going to convert this to draft for now.'
willcl-ark (member)objection, approachREAD-mode readers could observe stale or partially applied data if a writer crashed leaving an unapplied WALanswered · author converted PR to draft to work on non-blocking parallel reader accessansweredyes2026-07-07: 'since READ-mode methods then read the data files directly, a reader could observe stale or partially applied data if log_flag.dat is present from a failed writer. Is that OK in this use-case?'
Status from: 2026-07-09: sedited: 'Going to convert this to draft for now.'
pzafonte (contributor)objection, correctnessA second BlockTreeStore gets Success without locking, and destroying either instance releases the lock the other relies onopenchangedno2026-10-01: 'A second `BlockTreeStore` gets `Success` without locking anything, and destroying either of them releases the lock the other relies on. I think LockDirectory should fail on re-lock.'

Support:

  • w0xlt (contributor) (Approach ACK, 2025-05-06) [read]: The codebase changes seem surprisingly small for this proposal. Changing the code to reduce the dependency on LevelDB sounds good to me.
  • josibake (member) (Concept ACK, 2025-05-07) [read]: Allows parallel index building in tools like Electrs and Esplora via libbitcoinkernel without requiring Bitcoin Core to be stopped
  • theuni (member) (Concept ACK, 2025-05-15) [read]: Interested in replacing the LevelDB dependency for block and header metadata
  • ismaelsadeeq (member) (Concept ACK, 2025-06-03) [both]: Excited about replacing LevelDB dependency in kernel and tested on signet
  • marcofleon (member) (Concept ACK, 2025-06-05) [tested]: Differentially fuzzed BlockTreeDB and BlockTreeStore for ~5000 cpu hours with no issues

Participants:

  • w0xlt (contributor, support, 9 statements 2025-05-06 to 2026-06-10): Approach ACK; noted the small footprint and welcomed reducing the LevelDB dependency
  • Sjors (member, neutral, 3 statements 2025-05-07 to 2025-05-08): suggested considering a one-block-per-file architecture instead
  • josibake (member, support, 18 statements 2025-05-07 to 2026-06-11): Concept ACK; noted it unblocks parallel indexing in tools like Electrs and Esplora via libbitcoinkernel
  • mzumsande (member, neutral, 1 statements 2025-05-07 to 2025-05-07): commented on corruption causes and why reindex might still be needed
  • ryanofsky (member, objection, 2 statements 2025-05-07 to 2025-05-08): raised concern about data corruption on interrupted writes without checksums or atomic updates
  • shahsb (none, question, 1 statements 2025-05-07 to 2025-05-07): asked a series of high-level architectural and risk questions
  • l0rinc (member, neutral, 31 statements 2025-05-14 to 2026-06-17): benchmarked reindexing performance and provided extensive inline code review
  • hodlinator (member, neutral, 4 statements 2025-05-15 to 2026-09-18): discussed directory sharding schemes and noted potential synergy with PRESYNC header caching
  • sipa (member, neutral, 1 statements 2025-05-15 to 2025-05-15): suggested file hierarchy layout options
  • theuni (member, support, 2 statements 2025-05-15 to 2025-05-16): Concept ACK; supported moving block/header storage out of LevelDB
  • davidgumberg (member, question, 1 statements 2025-05-16 to 2025-05-16): asked about filesystem limits and DoS risk under hash-sharded directory layouts
  • maflcko (member, neutral, 1 statements 2025-05-16 to 2025-05-16): encouraged robust corruption detection on startup
  • ismaelsadeeq (member, support, 2 statements 2025-06-03 to 2025-12-22): Concept ACK; tested on signet
  • marcofleon (member, support, 2 statements 2025-06-05 to 2026-06-09): Concept ACK; differentially fuzzed BlockTreeDB vs BlockTreeStore for ~5000 CPU hours
  • stickies-v (member, objection, 60 statements 2026-01-07 to 2026-07-02): raised objection about directory syncing being a no-op on Windows, plus extensive testing and code review
  • HowHsu (contributor, neutral, 4 statements 2026-03-06 to 2026-03-06): participated in earlier review rounds
  • craigraw (contributor, neutral, 5 statements 2026-03-12 to 2026-03-12): participated in discussion
  • willcl-ark (member, objection, 18 statements 2026-06-02 to 2026-07-07): raised concerns about process locking and READ mode exposing torn state from unapplied WALs
  • janb84 (member, neutral, 5 statements 2026-06-03 to 2026-06-11): participated in earlier review rounds
  • alexanderwiederin (member, neutral, 10 statements 2026-06-09 to 2026-06-11): participated in review
  • yuvicc (contributor, neutral, 2 statements 2026-06-10 to 2026-06-12): participated in review
  • edilmedeiros (contributor, neutral, 1 statements 2026-06-16 to 2026-06-16): participated in review
  • stringintech (contributor, support, 2 statements 2026-06-29 to 2026-06-30): proposed dedicated writer directory lock and tests to fail fast on multiple writers
  • pzafonte (contributor, objection, 2 statements 2026-10-01 to 2026-10-01): reported that LockDirectory returns Success on re-lock and destroying either instance releases the lock the other relies on

Dependencies

Enables:

  • Parallel block data reading for external libbitcoinkernel consumers

Files

824 lines under test/bench/ci.

  • src/kernel/blocktreestorage.cpp +622/-0
  • src/test/blocktreestorage_tests.cpp +497/-0
  • src/kernel/blocktreestorage.h +209/-0
  • src/node/blockstorage.cpp +110/-47
  • test/functional/feature_blocktree_migration.py +105/-0
  • src/bench/write_block_index.cpp +99/-0
  • src/node/blockstorage.h +5/-46
  • src/test/fuzz/block_index.cpp +16/-16
  • src/chain.h +25/-0
  • test/functional/tool_bitcoin_chainstate.py +25/-0
  • src/kernel/bitcoinkernel.cpp +5/-16
  • src/test/kernel/test_kernel.cpp +8/-13
  • src/init.cpp +5/-11
  • test/functional/feature_init.py +8/-3
  • src/kernel/bitcoinkernel.h +0/-10
  • src/test/blockmanager_tests.cpp +2/-8
  • src/test/util/setup_common.cpp +2/-7
  • src/kernel/caches.h +1/-6
  • src/test/validation_chainstatemanager_tests.cpp +1/-6
  • src/kernel/bitcoinkernel_wrapper.h +0/-5
  • src/kernel/blockmanager_opts.h +2/-1
  • test/functional/feature_coinstatsindex_compatibility.py +3/-0
  • doc/files.md +1/-1
  • src/node/blockmanager_args.cpp +0/-2
  • src/test/util/setup_common.h +0/-2
  • src/validation.cpp +1/-1
  • src/CMakeLists.txt +1/-0
  • src/bench/CMakeLists.txt +1/-0
  • src/kernel/CMakeLists.txt +1/-0
  • src/test/CMakeLists.txt +1/-0
  • test/functional/test_runner.py +1/-0

Card

The short text the ranking pass compares across PRs.

PR #32427 replaces the LevelDB-based block tree database with a flat-file store backed by a write-ahead log and CRC32C checksums. The change solves a major limitation in libbitcoinkernel where LevelDB's exclusive process lock prevents secondary tools (like Electrs) from reading block and header indexes while bitcoind is running. Concept support is broad, but the PR is currently paused in draft while the author addresses WAL synchronization overhead and true parallel reader access raised by willcl-ark and stickies-v.

Data

dossier JSON · extract JSON · model ledger, generated 2026-10-02T01:24, confidence high, input hash b402c15f63eddf40
thread read through 2026-10-01 (269 statements), head 50133ca57b
labels: UTXO Db and Indexes, Needs rebase · draft

Categories

Kernel (libbitcoinkernel)

P1 · new feature (assessed alone as P2; Raised to P1: LevelDB's exclusive process lock is the concrete reason external consumers (Electrs/Esplora) cannot read block/header data from a running node, which is the category's defining use case.)

Enables external applications building on libbitcoinkernel to access block and header data concurrently while Bitcoin Core is running. Commenters note this unblocks parallel index building for downstream tools like Electrs and Esplora while reducing kernel dependencies on LevelDB.

Membership: Implements BlockTreeStore in src/kernel/ and updates bitcoinkernel interfaces to remove LevelDB locking restrictions.

Factors: security/stability 1, bug 0, performance 1, user value 2, leverage 2

Validation

P2 · new feature

Replaces the underlying LevelDB database for block index records with a custom WAL and flat-file format, directly affecting node crash safety, migration on startup, and file locking. Because this substantively alters validation persistence alongside kernel library capabilities, it warrants rigorous validation scrutiny regarding data corruption and power loss handling.

Membership: Labeled 'UTXO Db and Indexes' and replaces the core CBlockTreeDB storage engine with a custom WAL and flat-file store.

Factors: security/stability 1, bug 0, performance 1, user value 2, leverage 2