#36603 Improve loadtxoutset performance

Replaces the sequential assumeutxo snapshot loader with a multithreaded pipeline that reads, hashes, serializes, and writes UTXO entries directly into LevelDB. Staging snapshot data through CCoinsViewCache and per-batch fsync calls are eliminated in favor of committing the chainstate directory once loading completes. The PR also adds a txoutsetsummary subcommand to bitcoin-util for inspecting snapshot statistics, serialization hashes, and MuHash commitments offline.

Problem: Loading an assumeutxo snapshot with loadtxoutset is slow and memory-heavy because snapshot data is staged in CCoinsViewCache up to -dbcache limits, gated by repeated fsync writes, and hashed sequentially after LevelDB population finishes.

Evidence of importance:

• Author description: 'Significantly improves loadtxoutset performance via a number of changes: parallel threads for decoding, hashing, serializing and writing to leveldb; no fsync() gating writes; no redundant loading into CoinsView.'
• Author description: 'On my systems, performance ends up bottlenecked by leveldb CPU usage.'
• Commit 49aa42ccaa: 'The dump is no longer staged in a CCoinsViewCache and then hashed and flushed out of it, so -dbcache worth of cached coins during a mainnet load are gone along with the flush cycles. In addition, the hashing of the dump takes place in parallel with the import, rather than after it completes.'

Scope: Adds a reusable thread-safe BlockingQueue utility (src/util/blockingqueue.h) and a nosync mode to CDBWrapper that wraps LevelDB writable files to suppress synchronous disk flushes during bulk population. The largest files implementing the multithreaded import pipeline and bitcoin-util tool are truncated in the diff.

Risk of a mistake: high (validation). Flaws in outpoint ordering or direct-to-disk LevelDB batch writing could silently corrupt the assumeutxo chainstate, causing consensus validation failures or forks.

Changed since the previous assessment: No structural changes to the design or commit sequence; contains rebases and minor test refinements.

What the assessment lacked:

• rest_of_diff
• benchmark_numbers

Ready

Nothing known that would invalidate a review now.

Waiting on (from the thread read):

• reviewer: l0rinc to review the author's response on AssumeUTXO complexity and load performance 2026-10-23

Mild

objection that needs a change was answered, no reply from the objector (l0rinc: Adds AssumeUTXO-specific complexity and pollutes the codebase with multiple chainstates, making consensus-critical paths)

Thread notes: Reviewed author's updated performance benchmark comment; claims and participant status remain unchanged.

Objections, suggestions, and questions:

Participants:

• l0rinc (member, objection, 1 statements 2026-10-23 to 2026-10-23): objected to adding AssumeUTXO complexity to consensus-critical paths and questioned the motivation

Enables:

• #35054

257 lines under test/bench/ci.

• src/validation.cpp +120/-286
• src/utxo_snapshot.h +305/-0
• src/node/utxo_snapshot.cpp +244/-13
• src/bitcoin-util.cpp +137/-0
• src/util/blockingqueue.h +135/-0
• src/test/util_tests.cpp +128/-0
• src/node/utxo_snapshot.h +34/-91
• src/test/validation_chainstatemanager_tests.cpp +61/-1
• src/dbwrapper.cpp +51/-0
• src/txdb.h +31/-0
• src/test/coins_tests.cpp +3/-24
• src/txdb.cpp +20/-0
• src/util/fs_helpers.cpp +16/-0
• src/validation.h +1/-15
• src/coins.cpp +0/-10
• src/test/fuzz/utxo_snapshot.cpp +4/-6
• src/coins.h +0/-9
• src/test/coinsviewoverlay_tests.cpp +6/-3
• test/functional/feature_assumeutxo.py +4/-3
• src/dbwrapper.h +6/-0
• src/rpc/blockchain.cpp +3/-3
• src/test/util/chainstate.h +4/-2
• src/util/fs_helpers.h +6/-0
• src/test/fuzz/deserialize.cpp +2/-3
• src/node/blockstorage.cpp +1/-1
• src/test/fuzz/coins_view.cpp +1/-1
• test/lint/lint-circular-dependencies.py +0/-1

The short text the ranking pass compares across PRs.

Accelerates assumeutxo snapshot imports in loadtxoutset by replacing the sequential loader with a concurrent pipeline that reads, hashes, serializes, and writes coins directly into LevelDB. It removes CCoinsViewCache staging, avoids intermediate fsync calls during bulk writing, and introduces an offline txoutsetsummary tool in bitcoin-util to verify snapshot statistics and MuHash values. Currently, snapshot loading is serialized, bottlenecked by cache flush cycles, and calculates hashes only after population completes. The author notes that this pipeline shifts the performance bottleneck entirely to LevelDB CPU consumption. Work on P2P UTXO set sharing (#35054) builds on these changes.

P2 · new feature (assessed alone as P3; Raised to P2 and moved up: substantial snapshot-load performance gain plus an offline bitcoin-util inspection tool, and #35054 (P2P UTXO set sharing) builds on it.)

The PR adds a txoutsetsummary subcommand to bitcoin-util for inspecting snapshot statistics, serialization hashes, and MuHash commitments offline. This provides a useful offline inspection tool for operators and developers handling assumeutxo snapshots.

Membership: src/bitcoin-util.cpp (+137/-0) adds an offline txoutsetsummary tool.

Factors: security/stability 1, bug 0, performance 3, user value 2, leverage 2

P3 · new feature (ranking pass: Moved up: measurable snapshot-load speedup plus a reusable concurrency primitive that #35054 builds on.)

Adds a reusable thread-safe BlockingQueue utility in src/util/blockingqueue.h with dedicated unit tests. While created to support the pipeline, it constitutes a new general concurrency primitive for node utilities.

Membership: src/util/blockingqueue.h (+135/-0) introduces a general-purpose synchronization utility.

Factors: security/stability 1, bug 0, performance 3, user value 2, leverage 2

P2 · speedup

Significantly accelerates assumeutxo snapshot loading by replacing sequential loading with a parallelized pipeline and direct LevelDB writes. It removes the intermediate CCoinsViewCache staging and redundant flush cycles, addressing a major bottleneck in node bootstrap performance.

Membership: Substantially alters assumeutxo snapshot loading, chainstate LevelDB writing, and UTXO coins handling across src/validation.cpp, src/node/utxo_snapshot.cpp, and src/txdb.cpp.

Factors: security/stability 1, bug 0, performance 3, user value 2, leverage 2