#35581 node: add block template manager and track waitNext fee inflow
| Reviewability | Reviews | Agreement | Size | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| #35581 node: add block template manager and track waitNext fee inflow | Needs rebase | 0 | Strong | 752 + 1,416 tests | ||||||
|
|
|
|
| ||||||
Reviewability: ReadyStalePaused Agreement: StrongPositivePositive w/ caveatsNeutralMildDisputedBlockedCrickets Reviews: current code-review ACKs, then (+stale ACKs) and -NACKs; greener = more ACKs. Size: lines added or modified outside tests, then in tests; greener = smaller. | ||||||||||
Goal
Consolidates block template creation, waiting, and submission helpers into a `BlockTemplateManager` class registered with validation signals. Adds a `MempoolUpdated` validation interface event that tracks before-and-after fee rate diagram chunks with transaction references. Leverages these diagram snapshots to monitor mempool fee inflow per active template, replacing the polling loop in `waitNext` that assembled full blocks on each tick.
Problem: To check whether newly arriving transactions justify issuing a fresh block template, `waitNext` in the IPC mining interface repeatedly acquires `cs_main` and builds complete block templates solely to sum and compare their fees. Additionally, mining utility functions are free functions scattered across `miner.cpp` requiring manual threading of dependencies, and `SubmitBlock` allocates a throwaway validation listener on every block submitted.
Evidence of importance:
- PR description: 'Currently, to decide whether mempool fee inflow rose enough to be worth a new template, waitNext locks cs_main and assembles a full block template every tick just to sum its fees and compare. After cluster mempool... we shouldn\'t have to build a whole block template to answer that question.'
- PR description links to Delving Bitcoin research: 'determining-blocktemplate-fee-increase-using-fee-rate-diagram/2052'.
- Author notes the fuzz harness for the new manager uncovered a miner chunk-limit check bug fixed in merged PR #35580.
Scope: About half of the patch (+1090 lines) consists of tests, including a new fuzz harness for `BlockTemplateManager` and unit tests in `mempool_update_tests.cpp`. The manager refactor was originally split into #35675 (now merged), and this branch retains the commits up through fee-inflow tracking.
Risk of a mistake: medium (mempool_policy, validation, tests). Mistakes in mempool chunk fee tracking or validation signal ordering could cause mining clients to stall, miss profitable transactions, or encounter deadlocks.
Changed since the previous assessment: The prior dependency #35675 was merged upstream; this branch remains to be rebased on top of it.
Reviewability
Ready · Needs rebase
Conflicts with master; review the logic now, the rebase is mechanical.
Waiting on (from the thread read):
- author: address comments and update PR after opening the standalone BlockTemplateManager PR #35675 2026-07-01
- author: address comments and extract BlockTemplateManager into dedicated PR 2026-07-01
Agreement
Strong
substantive support, no open objection (pablomartin4btc)
Thread notes: Reclassified Sjors' and pablomartin4btc's review comments as suggestions since neither stated a concrete harm of merging as-is. Added missing suggestion rc:3506167320. / Reclassified Sjors's PR split request and pablomartin4btc's precondition doc request from objections to suggestions under the current rules, since neither names a concrete cost of merging as-is. Added pablomartin4btc's suggestion on restoring the miner comment.
Objections, suggestions, and questions:
| Reviewer | Type | Harm, request, or question | Status | Must be | Author replied | Quote |
|---|---|---|---|---|---|---|
| Sjors (member) | suggestion, scope | split BlockTemplateManager into its own focused PR with waitNext cluster mempool enhancement as a followup | answered · author agreed to split the PR | answered | yes | 2026-06-25: 'It would be nice to have a PR focussed on the BlockTemplateManager... The `waitNext()` cluster mempool enhancement would be a natural followup.' Status from: 2026-07-01: ismaelsadeeq: 'I will address these comments when I open up a PR that adds the block template manager alone, as @Sjors suggested' |
| pablomartin4btc (member) | suggestion, maintenance | document implicit precondition on IsStale that options must match those used when template_id was created | answered · author noted fixed in split PR #35675 | answered | yes | 2026-07-01: 'IsStale is a public method but has an implicit precondition: options must match the ones used when `template_id` was created, otherwise it silently returns false. Could a comment document this (if appropriate)?' Status from: 2026-07-07: ismaelsadeeq: 'Fixed in #35675' |
| pablomartin4btc (member) | suggestion, maintenance | restore and update comment explaining why staleness is checked cheaply while rebuild is once per second | open · author deferred review comments to split PR #35675 | answered | no | 2026-07-01: 'There was a comment there in the old code that was removed as I think part of it was no longer accurate. Perhaps it can be add:' |
Support:
- w0xlt (contributor) (Concept ACK, 2026-06-22): (no reason given) [not substantive]
- pablomartin4btc (member) (Concept ACK, 2026-07-01) [read: src/node/block_template_manager.cpp]: The approach of moving free functions onto a class that holds state plus incremental snapshot tracking to avoid full template rebuilds seems clean. The fuzz test catching an actual bug (#35580) is a strong signal.
Participants:
- w0xlt (contributor, support, 1 statements 2026-06-22 to 2026-06-22): Concept ACK
- Sjors (member, neutral, 1 statements 2026-06-25 to 2026-06-25): suggested splitting BlockTemplateManager into its own PR
- pablomartin4btc (member, support, 3 statements 2026-07-01 to 2026-07-01): Concept ACK; suggested documenting a precondition and restoring an updated comment
Files
1474 lines under test/bench/ci.
- src/test/fuzz/block_template_manager.cpp +556/-0
- src/test/mempool_update_tests.cpp +535/-0
- src/node/block_template_manager.cpp +245/-46
- src/test/miner_tests.cpp +259/-29
- src/txmempool.cpp +149/-41
- src/node/block_template_manager.h +139/-10
- src/txgraph.cpp +47/-17
- src/test/fuzz/txgraph.cpp +23/-12
- src/node/interfaces.cpp +24/-4
- src/txgraph.h +21/-4
- src/validationinterface.cpp +24/-0
- src/test/util/setup_common.cpp +19/-0
- src/util/hasher.cpp +19/-0
- src/kernel/mempool_entry.h +18/-0
- src/node/miner.h +15/-3
- src/policy/packages.cpp +4/-14
- src/test/blockencodings_tests.cpp +9/-4
- src/txmempool.h +9/-2
- src/policy/fees/mempool_estimator.cpp +8/-2
- src/node/miner.cpp +8/-1
- src/validationinterface.h +9/-0
- src/init.cpp +7/-0
- src/util/hasher.h +4/-0
- test/functional/interface_zmq.py +2/-2
- src/test/fuzz/tx_pool.cpp +1/-2
- src/bench/mempool_stress.cpp +1/-1
- src/test/fuzz/connect_block.cpp +1/-1
- src/test/fuzz/utxo_snapshot.cpp +1/-1
- src/test/mempool_tests.cpp +1/-1
- src/test/rbf_tests.cpp +1/-1
- src/test/util/setup_common.h +2/-0
- src/test/util/validation.cpp +1/-1
- src/test/validation_chainstatemanager_tests.cpp +1/-1
- src/validation.cpp +1/-1
- src/wallet/test/fuzz/fees.cpp +1/-1
- src/node/mining_types.h +1/-0
- src/test/CMakeLists.txt +1/-0
- src/test/fuzz/CMakeLists.txt +1/-0
- src/test/fuzz/package_eval.cpp +0/-1
Card
The short text the ranking pass compares across PRs.
Introduces a BlockTemplateManager and adds a MempoolUpdated validation interface signal carrying fee rate diagram chunk changes. Currently, waitNext repeatedly acquires cs_main and constructs full block templates every tick solely to determine if fee increases justify a new template. By tracking fee-rate diagram deltas across mempool updates, waitNext detects staleness without redundant block assembly. Mining helper functions and block submission are also centralized into the manager.
Data
thread read through 2026-07-07 (7 statements), head 24a87b7869
labels: Mining, Needs rebase, IPC · draft
Categories
Mempool and policy
P3 · new feature
Adds incremental feerate diagram snapshot diffing to `txmempool` and `txgraph` when transactions enter or leave the pool. While driven by mining needs, it introduces a useful notification mechanism for pool state changes without altering core policy or acceptance rules.
Membership: Modifies txmempool and txgraph to compute and emit before-and-after fee rate diagram chunks upon mempool updates.
Factors: security/stability 1, bug 0, performance 2, user value 1, leverage 1
Mining
P2 · speedup (ranking pass: Root of the BlockTemplateManager chain (#33922, #36356, #36357 build on it) as well as a measurable CPU/lock-contention win for template polling.)
Replaces the polling loop in `waitNext` that assembled complete block templates and acquired `cs_main` on every tick with incremental fee-rate diagram tracking. This avoids significant CPU overhead and lock contention for miners waiting on template updates.
Membership: Carries the Mining label and redesigns block template creation, waitNext, and fee inflow tracking under BlockTemplateManager.
Factors: security/stability 1, bug 0, performance 2, user value 1, leverage 1