#35581 node: add block template manager and track waitNext fee inflow

ReviewabilityReviewsAgreementSize
#35581 ismaelsadeeq node: add block template manager and track waitNext fee inflowNeeds rebase0Strong752 + 1,416 tests
  • Avoid rebuilding full block templates repeatedly just to check if mempool fee increases warrant a new one
  • Reduce CPU overhead and lock contention for miners waiting on template updates
  • Worth reviewing now despite conflict with master, as the needed rebase is mechanical
  • Strong support with no blocking objections
  • Supports incremental tracking over full rebuilds and notes fuzz test found a real bug (pablomartin4btc)
  • Concept approval without stated reasons (w0xlt)
  • Open suggestion to restore and update a miner comment on staleness checks (pablomartin4btc)
  • 752 lines added or modified outside tests
  • 1,416 lines added or modified in tests
  • 587 lines removed in total
  • 51 files, 27 commits
Mempool and policy#12 of 14
P3 · new feature
  • P3 because it adds incremental feerate diagram snapshots without changing core mempool policy
  • Provides a useful state-change notification mechanism driven primarily by mining needs
Mining#2 of 17
P2 · speedup
  • P2 because it stops full block assembly and lock acquisition on every tick during template polling
  • Significantly cuts CPU usage and lock contention for miners waiting on template updates
  • Ranking pass: Root of the BlockTemplateManager chain (#33922, #36356, #36357 build on it) as well as a measurable CPU/lock-contention win for template polling.
Reviewability: ReadyStalePaused
Agreement: StrongPositivePositive w/ caveatsNeutralMildDisputedBlockedCrickets
Reviews: current code-review ACKs, then (+stale ACKs) and -NACKs; greener = more ACKs.
Size: lines added or modified outside tests, then in tests; greener = smaller.

Goal

Consolidates block template creation, waiting, and submission helpers into a `BlockTemplateManager` class registered with validation signals. Adds a `MempoolUpdated` validation interface event that tracks before-and-after fee rate diagram chunks with transaction references. Leverages these diagram snapshots to monitor mempool fee inflow per active template, replacing the polling loop in `waitNext` that assembled full blocks on each tick.

Problem: To check whether newly arriving transactions justify issuing a fresh block template, `waitNext` in the IPC mining interface repeatedly acquires `cs_main` and builds complete block templates solely to sum and compare their fees. Additionally, mining utility functions are free functions scattered across `miner.cpp` requiring manual threading of dependencies, and `SubmitBlock` allocates a throwaway validation listener on every block submitted.

Evidence of importance:

  • PR description: 'Currently, to decide whether mempool fee inflow rose enough to be worth a new template, waitNext locks cs_main and assembles a full block template every tick just to sum its fees and compare. After cluster mempool... we shouldn\'t have to build a whole block template to answer that question.'
  • PR description links to Delving Bitcoin research: 'determining-blocktemplate-fee-increase-using-fee-rate-diagram/2052'.
  • Author notes the fuzz harness for the new manager uncovered a miner chunk-limit check bug fixed in merged PR #35580.

Scope: About half of the patch (+1090 lines) consists of tests, including a new fuzz harness for `BlockTemplateManager` and unit tests in `mempool_update_tests.cpp`. The manager refactor was originally split into #35675 (now merged), and this branch retains the commits up through fee-inflow tracking.

Risk of a mistake: medium (mempool_policy, validation, tests). Mistakes in mempool chunk fee tracking or validation signal ordering could cause mining clients to stall, miss profitable transactions, or encounter deadlocks.

Changed since the previous assessment: The prior dependency #35675 was merged upstream; this branch remains to be rebased on top of it.

Reviewability

Ready · Needs rebase

Conflicts with master; review the logic now, the rebase is mechanical.

Waiting on (from the thread read):

  • author: address comments and update PR after opening the standalone BlockTemplateManager PR #35675 2026-07-01
  • author: address comments and extract BlockTemplateManager into dedicated PR 2026-07-01

Agreement

Strong

substantive support, no open objection (pablomartin4btc)

Thread notes: Reclassified Sjors' and pablomartin4btc's review comments as suggestions since neither stated a concrete harm of merging as-is. Added missing suggestion rc:3506167320. / Reclassified Sjors's PR split request and pablomartin4btc's precondition doc request from objections to suggestions under the current rules, since neither names a concrete cost of merging as-is. Added pablomartin4btc's suggestion on restoring the miner comment.

Objections, suggestions, and questions:

ReviewerTypeHarm, request, or questionStatusMust beAuthor repliedQuote
Sjors (member)suggestion, scopesplit BlockTemplateManager into its own focused PR with waitNext cluster mempool enhancement as a followupanswered · author agreed to split the PRansweredyes2026-06-25: 'It would be nice to have a PR focussed on the BlockTemplateManager... The `waitNext()` cluster mempool enhancement would be a natural followup.'
Status from: 2026-07-01: ismaelsadeeq: 'I will address these comments when I open up a PR that adds the block template manager alone, as @Sjors suggested'
pablomartin4btc (member)suggestion, maintenancedocument implicit precondition on IsStale that options must match those used when template_id was createdanswered · author noted fixed in split PR #35675answeredyes2026-07-01: 'IsStale is a public method but has an implicit precondition: options must match the ones used when `template_id` was created, otherwise it silently returns false. Could a comment document this (if appropriate)?'
Status from: 2026-07-07: ismaelsadeeq: 'Fixed in #35675'
pablomartin4btc (member)suggestion, maintenancerestore and update comment explaining why staleness is checked cheaply while rebuild is once per secondopen · author deferred review comments to split PR #35675answeredno2026-07-01: 'There was a comment there in the old code that was removed as I think part of it was no longer accurate. Perhaps it can be add:'

Support:

  • w0xlt (contributor) (Concept ACK, 2026-06-22): (no reason given) [not substantive]
  • pablomartin4btc (member) (Concept ACK, 2026-07-01) [read: src/node/block_template_manager.cpp]: The approach of moving free functions onto a class that holds state plus incremental snapshot tracking to avoid full template rebuilds seems clean. The fuzz test catching an actual bug (#35580) is a strong signal.

Participants:

  • w0xlt (contributor, support, 1 statements 2026-06-22 to 2026-06-22): Concept ACK
  • Sjors (member, neutral, 1 statements 2026-06-25 to 2026-06-25): suggested splitting BlockTemplateManager into its own PR
  • pablomartin4btc (member, support, 3 statements 2026-07-01 to 2026-07-01): Concept ACK; suggested documenting a precondition and restoring an updated comment

Files

1474 lines under test/bench/ci.

  • src/test/fuzz/block_template_manager.cpp +556/-0
  • src/test/mempool_update_tests.cpp +535/-0
  • src/node/block_template_manager.cpp +245/-46
  • src/test/miner_tests.cpp +259/-29
  • src/txmempool.cpp +149/-41
  • src/node/block_template_manager.h +139/-10
  • src/txgraph.cpp +47/-17
  • src/test/fuzz/txgraph.cpp +23/-12
  • src/node/interfaces.cpp +24/-4
  • src/txgraph.h +21/-4
  • src/validationinterface.cpp +24/-0
  • src/test/util/setup_common.cpp +19/-0
  • src/util/hasher.cpp +19/-0
  • src/kernel/mempool_entry.h +18/-0
  • src/node/miner.h +15/-3
  • src/policy/packages.cpp +4/-14
  • src/test/blockencodings_tests.cpp +9/-4
  • src/txmempool.h +9/-2
  • src/policy/fees/mempool_estimator.cpp +8/-2
  • src/node/miner.cpp +8/-1
  • src/validationinterface.h +9/-0
  • src/init.cpp +7/-0
  • src/util/hasher.h +4/-0
  • test/functional/interface_zmq.py +2/-2
  • src/test/fuzz/tx_pool.cpp +1/-2
  • src/bench/mempool_stress.cpp +1/-1
  • src/test/fuzz/connect_block.cpp +1/-1
  • src/test/fuzz/utxo_snapshot.cpp +1/-1
  • src/test/mempool_tests.cpp +1/-1
  • src/test/rbf_tests.cpp +1/-1
  • src/test/util/setup_common.h +2/-0
  • src/test/util/validation.cpp +1/-1
  • src/test/validation_chainstatemanager_tests.cpp +1/-1
  • src/validation.cpp +1/-1
  • src/wallet/test/fuzz/fees.cpp +1/-1
  • src/node/mining_types.h +1/-0
  • src/test/CMakeLists.txt +1/-0
  • src/test/fuzz/CMakeLists.txt +1/-0
  • src/test/fuzz/package_eval.cpp +0/-1

Card

The short text the ranking pass compares across PRs.

Introduces a BlockTemplateManager and adds a MempoolUpdated validation interface signal carrying fee rate diagram chunk changes. Currently, waitNext repeatedly acquires cs_main and constructs full block templates every tick solely to determine if fee increases justify a new template. By tracking fee-rate diagram deltas across mempool updates, waitNext detects staleness without redundant block assembly. Mining helper functions and block submission are also centralized into the manager.

Data

dossier JSON · extract JSON · model ledger, generated 2026-09-27T01:24, confidence high, input hash 2c0531eb28073d02
thread read through 2026-07-07 (7 statements), head 24a87b7869
labels: Mining, Needs rebase, IPC · draft

Categories

Mempool and policy

P3 · new feature

Adds incremental feerate diagram snapshot diffing to `txmempool` and `txgraph` when transactions enter or leave the pool. While driven by mining needs, it introduces a useful notification mechanism for pool state changes without altering core policy or acceptance rules.

Membership: Modifies txmempool and txgraph to compute and emit before-and-after fee rate diagram chunks upon mempool updates.

Factors: security/stability 1, bug 0, performance 2, user value 1, leverage 1

Mining

P2 · speedup (ranking pass: Root of the BlockTemplateManager chain (#33922, #36356, #36357 build on it) as well as a measurable CPU/lock-contention win for template polling.)

Replaces the polling loop in `waitNext` that assembled complete block templates and acquired `cs_main` on every tick with incremental fee-rate diagram tracking. This avoids significant CPU overhead and lock contention for miners waiting on template updates.

Membership: Carries the Mining label and redesigns block template creation, waitNext, and fee inflow tracking under BlockTemplateManager.

Factors: security/stability 1, bug 0, performance 2, user value 1, leverage 1