#32554 bench: replace embedded raw block with configurable block generator
| Reviewability | Reviews | Agreement | Size | ||||
|---|---|---|---|---|---|---|---|
| #32554 bench: replace embedded raw block with configurable block generator | Ready | 0 | Strong | 0 + 579 tests | |||
|
|
|
|
| |||
Reviewability: ReadyStalePaused Agreement: StrongPositivePositive w/ caveatsNeutralMildDisputedBlockedCrickets Reviews: current code-review ACKs, then (+stale ACKs) and -NACKs; greener = more ACKs. Size: lines added or modified outside tests, then in tests; greener = smaller. | |||||||
Goal
Replaces the embedded raw block fixture `block413567.raw` with a programmatic regtest block generator for benchmarks. The generator deterministically constructs consensus-valid blocks according to configurable script recipes, such as legacy pre-SegWit mixes or modern Taproot and witness mixes. It also generates synthetic block undo data to support block-to-JSON serialization benchmarks that inspect input fees and prevouts.
Problem: Block benchmarks relied on an embedded ~1 MB binary fixture mined in 2016 before SegWit and Taproot. Its fixed legacy transaction mix prevented benchmarking modern witness script workloads or varying transaction compositions, and JSON benchmarks could not exercise input fee and prevout paths without undo data.
Evidence of importance:
- PR description: 'Benchmarks embed block 413567, a nearly 1 MB binary fixture mined in 2016 before SegWit v0 and Taproot. Its transaction and script mix is fixed. Replacing it with a newer block, as in #32457, would retain a binary fixture and a single fixed workload.'
- Commit 01f023b61d: 'Without undo data, the block-to-JSON benchmarks skip fee and prevout work at detail verbosity.'
Scope: Touches benchmark infrastructure and unit tests only, removing `src/bench/data/block413567.raw` and adding `src/bench/block_generator.{h,cpp}`.
Risk of a mistake: low (tests). Only touches benchmark harness data generation and test files; errors would result in inaccurate benchmarks or test failures.
Changed since the previous assessment: No substantial changes to the approach; the diff and commit messages reflect the same implementation structure.
Reviewability
Ready
Nothing known that would invalidate a review now.
Agreement
Strong
substantive support, no open objection (yuvicc, Raimo33, hodlinator, 0xB10C)
Thread notes: Initial record. hodlinator raised an objection regarding blocks exceeding size limits during script experimentation, which was resolved after the author explained the deterministic seed design and added multi-seed sanity tests.
Objections, suggestions, and questions:
| Reviewer | Type | Harm, request, or question | Status | Must be | Author replied | Quote |
|---|---|---|---|---|---|---|
| hodlinator (member) | objection, correctness | the generator can produce blocks that exceed block size limits when transaction count or script sizes are large | fixed · fix pushed · v1, not yet re-checked | answered | yes | 2025-12-23: 'When experimenting with changing the scripts I hit block size limits with just 2000 transactions due to large scripts.' Status from: 2026-02-16: l0rinc: 'I would expect users to create a block by setting exact values or just a seed and experiment until they get one that fits their needs... I have added a `block_generator_multiple_seed_sanity` to help us discover these invalid setups anyway.' |
Support:
- laanwj (member) (Concept ACK, 2025-05-19): Concept ACK [not substantive]
- yuvicc (contributor) (Concept ACK, 2025-06-29): This one is pretty useful as I can test with up-to date test_data.
- Raimo33 (contributor) (Concept ACK, 2025-10-19): Concept ACK for more realistic benchmarks
- hodlinator (member) (Concept ACK, 2026-02-11): Strong problem ACK - It's quite bad that we are only benchmarking against a pre-segwit block.
- 0xB10C (member) (Concept ACK, 2026-09-16): something like this is the right move as a) it takes up far less repo space, and b) is extensible and we don't need to add a recent mainnet block every few years
Participants:
- laanwj (member, support, 1 statements 2025-05-19 to 2025-05-19): Concept ACK
- yuvicc (contributor, support, 1 statements 2025-06-29 to 2025-06-29): Concept ACK; noted it is useful for testing with up-to-date data
- Raimo33 (contributor, support, 1 statements 2025-10-19 to 2025-10-19): Concept ACK for more realistic benchmarks
- hodlinator (member, objection, 22 statements 2025-12-22 to 2026-02-11): Concept ACK, but noted generated blocks could exceed block size limits with large scripts
- maflcko (member, neutral, 1 statements 2026-02-20 to 2026-02-20): suggested drafting PR due to CI assertion failure
- 0xB10C (member, support, 1 statements 2026-09-16 to 2026-09-16): Concept ACK; noted runtime generation reduces repository size and is extensible
Files
632 lines under test/bench/ci.
- src/bench/block_generator.cpp +281/-0
- src/test/bench_block_generator_tests.cpp +150/-0
- src/bench/block_generator.h +96/-0
- src/bench/rpc_blockchain.cpp +23/-17
- src/bench/readwriteblock.cpp +13/-20
- src/bench/checkblock.cpp +6/-6
- src/bench/load_external.cpp +6/-5
- src/bench/CMakeLists.txt +2/-5
- src/test/CMakeLists.txt +2/-0
- src/bench/data/block413567.raw +None/-None
Card
The short text the ranking pass compares across PRs.
This PR replaces the embedded 1 MB binary fixture block413567.raw from 2016 with a deterministic runtime block and undo data generator for benchmarks. Existing benchmarks were locked into a pre-SegWit, pre-Taproot transaction mix, bloating the git tree with a raw binary file while preventing developers from benchmarking modern script workloads. The generator produces valid regtest blocks and matching synthetic undo data according to configurable script recipes, allowing benchmarks to cover both legacy and modern witness structures without requiring static chain data. This supersedes previous attempts to replace the fixture with a newer static block like PR 32457.
Data
thread read through 2026-09-16 (43 statements), head 01f023b61d
Discussion read under schema 1 rules: suggestions and questions may be missing, and an objection may combine several harms (schema history).
labels: Tests
Categories
Test infrastructure
P3 · test coverage (ranking pass: Moved down to sit just after #36254, which fixes the same misleading verbosity-3 benchmark with far less machinery; the generator is the broader but more deferrable option.)
Enables benchmarking modern SegWit and Taproot workloads by replacing an obsolete 2016 pre-SegWit binary block fixture with a programmatic generator. As noted in discussion, 'it's quite bad that we are only benchmarking against a pre-segwit block', making this a clear and valuable test framework improvement, though reasonably deferrable.
Membership: Matches 'Tests' label and modifies benchmark machinery in 'src/bench/' and test utilities in 'src/test/'.
Factors: security/stability 0, bug 0, performance 0, user value 0, leverage 1