#35581 node: add block template manager and track waitNext fee inflow
https://github.com/bitcoin/bitcoin/pull/35581 · · +2664/-587 in 51 files, 27 commits · labels: Mining, IPC · draft
Goal
- Avoid assembling full block templates on every tick just to check for mempool fee changes
- Reduce lock contention and CPU load during block generation for mining pools and template consumers
- Consolidate block creation and template state management into a unified interface
This PR introduces `BlockTemplateManager` to encapsulate block template generation, tip tracking, and submission state previously scattered across `src/node/miner.cpp`. It also utilizes cluster mempool fee-rate diagram changes to track fee inflow incrementally, replacing repeated full template rebuilds in `waitNext` with a lightweight staleness check.
Problem: Miners and mining pools polling `waitNext` currently trigger complete block assembly on every tick just to see if mempool fee increases warrant updating the template. In addition, mining and template state is scattered across free functions requiring repetitive parameter plumbing.
Category: Mining (#3 of 13)
P2 · speedup
- P2 because replacing full block assembly on every poll removes main lock contention and CPU load for pools
- Meaningfully improves performance and operation for Stratum v2 and mining template consumers
Noticeably improves mining template update efficiency by tracking mempool chunk fee inflow instead of rebuilding entire block templates repeatedly under `waitNext`. Sjors noted it unblocks memory load tracking in #33922.
Membership: Touches src/node/miner.cpp, introduces BlockTemplateManager, alters waitNext fee inflow detection, and carries the Mining label.
Factors: security/stability 1, bug 0, performance 2, user value 2, leverage 2
Category: Mempool and policy (#2 of 10)
P3 · new feature
- P3 because caching diagram chunks provides clean fee rate tracking hooks without holding the mempool lock
- Does not alter mempool acceptance policy or core data structures
Adds diagram chunk caching to ChangeSet and provides the MemPoolChunksUpdate notification mechanism used by downstream consumers to observe diagram changes asynchronously.
Membership: Modifies txmempool.cpp/h, txgraph.cpp/h, kernel/mempool_entry.h, and implements MempoolUpdated notifications with fee rate diagram caching.
Factors: security/stability 0, bug 0, performance 1, user value 0, leverage 2
Category: IPC / multiprocess (#16 of 20)
P3 · cleanup
- Cleans up IPC usage by letting in-process callers bypass the IPC mining interface wrapper and avoiding unnecessary defensive block copying.
Cleans up IPC usage by letting in-process callers bypass the IPC mining interface wrapper and avoiding unnecessary defensive block copying.
Membership: Touches src/node/interfaces.cpp and context.h, removing NodeContext::mining and routing in-process callers directly through BlockTemplateManager.
Factors: security/stability 0, bug 0, performance 1, user value 0, leverage 1
Category: Validation (#46 of 48)
P4 · cleanup
- The validation impact is purely incidental plumbing for the new validation interface notification signal and passing height to removeForBlock.
The validation impact is purely incidental plumbing for the new validation interface notification signal and passing height to removeForBlock.
Membership: Touches validationinterface.h/cpp adding MempoolUpdated signal and adjusts removeForBlock in validation.cpp.
Factors: security/stability 0, bug 0, performance 0, user value 0, leverage 0
Reviewability: Ready: Review #35675 first
- Review #35675 first
This PR is stacked on top of #35675, which extracts the core BlockTemplateManager class; reviewers should examine #35675 first.
Author status: Active; actively force-pushing rebased commits and maintaining the prerequisite PR #35675.
Resolved concerns:
- Sjors suggested splitting out the `BlockTemplateManager` introduction into its own PR so other work (such as #33922) can build on it; the author did so in #35675 and addressed related inline comments there.
- pablomartin4btc noted implicit preconditions on `IsStale` and missing comments, which the author confirmed addressed in #35675.
Agreement: Strong
- Strong support for incremental fee inflow tracking and architectural cleanup
- Suggested splitting out the base manager into a dedicated preparatory PR (Sjors)
- Concept approval praising incremental snapshot tracking and bug discovery via fuzzing (pablomartin4btc)
- Concept approval without stated reasons (w0xlt)
Strong concept support; Sjors requested splitting BlockTemplateManager into #35675, which author completed.
Multiple maintainers and contributors support the concept. The only process criticism was splitting the PR, which the author completed by creating #35675.
- w0xlt: Concept ACK (2026-06-22)
- pablomartin4btc: Concept ACK with positive review of the architecture (2026-07-01)
- Sjors: asked to split BlockTemplateManager into a separate PR (2026-07-01)
Objections:
| Reviewer | Kind | Harm | Status | Blocking | Author replied | Quote |
|---|---|---|---|---|---|---|
| Sjors | scope | combining BlockTemplateManager with the waitNext cluster mempool enhancement makes it harder to review and build dependent work on top | resolved | no | yes | 2026-06-25: "It would be nice to have a PR focussed on the BlockTemplateManager... The `waitNext()` cluster mempool enhancement would be a natural followup." Settled: 2026-07-01: "I will address these comments when I open up a PR that adds the block template manager alone, as @Sjors suggested" |
Support:
- w0xlt: Concept ACK [not substantive]
- pablomartin4btc: "The approach of moving free functions onto a class that holds state plus incremental snapshot tracking to avoid full template rebuilds seems clean. The fuzz test catching an actual bug (#35580) is a strong signal."
Participants: w0xlt (support), Sjors (objection), pablomartin4btc (support)
State derived from the lists: substantive support, no open objection (pablomartin4btc)
Review verdicts (DrahtBot): 0
- Concept ACK: w0xlt, pablomartin4btc
Dependencies
Depends on: #35675
Enables:
- #33922 mining: add getMemoryLoad() and track template non-mempool memory footprint
Based on (shares commits with): #35675
Files
1585 lines under test/bench/ci.
- src/test/fuzz/block_template_manager.cpp +556/-0
- src/test/mempool_update_tests.cpp +535/-0
- src/node/block_template_manager.cpp +487/-0
- src/test/miner_tests.cpp +278/-34
- src/node/miner.cpp +8/-244
- src/node/block_template_manager.h +235/-0
- src/txmempool.cpp +149/-41
- src/node/miner.h +15/-61
- src/rpc/mining.cpp +36/-38
- src/txgraph.cpp +47/-17
- src/node/interfaces.cpp +35/-27
- src/test/util/setup_common.cpp +34/-8
- src/test/fuzz/txgraph.cpp +23/-12
- src/rpc/blockchain.cpp +15/-16
- src/txgraph.h +21/-4
- src/validationinterface.cpp +24/-0
- src/init.cpp +15/-8
- src/test/validation_block_tests.cpp +10/-9
- src/util/hasher.cpp +19/-0
- src/kernel/mempool_entry.h +18/-0
- src/policy/packages.cpp +4/-14
- src/test/util/mining.cpp +9/-8
- src/node/context.h +3/-11
- src/test/blockencodings_tests.cpp +9/-4
- src/txmempool.h +9/-2
- src/policy/fees/mempool_estimator.cpp +8/-2
- src/test/peerman_tests.cpp +5/-4
- src/validationinterface.h +9/-0
- src/rpc/server_util.cpp +4/-4
- src/test/fuzz/rpc.cpp +4/-4
- src/rpc/server_util.h +2/-4
- src/test/util/setup_common.h +6/-0
- src/util/hasher.h +4/-0
- test/functional/interface_zmq.py +2/-2
- src/test/fuzz/connect_block.cpp +3/-0
- src/test/fuzz/tx_pool.cpp +1/-2
- src/test/fuzz/utxo_snapshot.cpp +3/-0
- src/test/util/validation.cpp +3/-0
- src/test/validation_chainstatemanager_tests.cpp +3/-0
- src/wallet/test/fuzz/fees.cpp +3/-0
- ci/test/03_test_script.sh +1/-1
- src/bench/mempool_stress.cpp +1/-1
- src/node/context.cpp +1/-1
- src/test/mempool_tests.cpp +1/-1
- src/test/rbf_tests.cpp +1/-1
- src/validation.cpp +1/-1
- src/CMakeLists.txt +1/-0
- src/node/mining_types.h +1/-0
- src/test/CMakeLists.txt +1/-0
- src/test/fuzz/CMakeLists.txt +1/-0
- src/test/fuzz/package_eval.cpp +0/-1
Card
This PR adds BlockTemplateManager to consolidate mining helper functions and template creation state, and leverages cluster mempool fee-rate diagrams to avoid repeated block assembly during waitNext. Mining clients querying for new templates benefit from reduced CPU overhead and lock contention. The base template manager refactor was split out into #35675, which reviewers should review first. Concept support is strong, with Sjors noting this will unblock template memory footprint tracking in #33922.