#31868 [IBD] specialize block serialization

• Reduce overhead when serializing single bytes and computing serialized data sizes
• Speed up block serialization and initial block download

This pull request optimizes serialization and deserialization routines by adding specialized handling for single-byte reads and writes in AutoFile, DataStream, VectorWriter, and CSHA256, alongside static extent spans. It also streamlines SizeComputer by short-circuiting byte emission when calculating serialized sizes and adds microbenchmarks for block serialization.

Problem: Generic stream serialization infrastructure incurs unnecessary overhead (such as small buffer allocations or memcpy calls) when writing single bytes, and SizeComputer unnecessarily evaluates endianness and byte emission logic when only computing sizes.

P4 · speedup

• P4 because real-world performance gains remain speculative
• Existing buffered streams already prevent single-byte writes during IBD, making gains marginal
• Adds template and code complexity without verified practical speedups

The PR attempts micro-optimizations across streams and serialization templates, but the real-world value remains speculative. As maflcko highlighted, block operations in IBD already pass through buffered streams, making the single-byte AutoFile specializations mostly ineffective in practice while adding template complexity.

Membership: Modifies shared serialization and stream primitives in src/serialize.h, src/streams.h, and src/crypto/sha256.h.

Factors: security/stability 0, bug 0, performance 1, user value 0, leverage 0

• Needs rebase due to merge conflicts
• Author is holding local revisions and has been inactive

The PR has merge conflicts, needs a rebase, and the author has been silent for over 60 days while holding local reworks.

Author status: silent since 2026-06-30; mentioned having a rebased branch locally to test once other work is finished

Open concerns:

• maflcko noted that during IBD, block reading and writing is already buffered by BufferedReader and BufferedWriter, so AutoFile single-byte writes are not actually hit in production.
• sedited could not reproduce the claimed performance improvements on their setup and questioned the necessity of single-byte specializations in crypto/sha256.
• The author acknowledged that the single-byte specializations add complexity and ugliness, keeping the PR in draft until cleaner alternatives and refreshed benchmarks can be evaluated.

Resolved concerns:

• theuni requested waiting for the std::span replacement (#31519) to land before specializing on extents; the PR was subsequently rebased after #31519 merged.

• Mild skepticism about real-world performance gains versus added complexity
• Questions whether production IBD hits these paths given existing buffered I/O (maflcko)
• Unable to reproduce claimed benchmark performance gains (sedited)
• Author acknowledged the added complexity and plans to retest local changes (l0rinc)

Mild skepticism from maflcko and sedited regarding real-world benefits versus code complexity

Reviewers have not blocked the PR, but both questioned whether the microbenchmark gains translate to real-world IBD benefits given existing buffered I/O, and the author concurred that the current code is too complex and needs remeasurement.

• maflcko asked what hits this during IBD since buffered reader/writers are already used: 'The Buffered* then passes them to AutoFile, which is modified here, but never receives 1-byte writes anymore'
• sedited commented: 'I am not really reproducing any of the performance changes on my machine yet'
• l0rinc acknowledged: 'Valid concerns. I have a rebased and updated branch locally; once I finish the reviews and PRs I'm working on, I'll retest and adjust this one'

Review verdicts (DrahtBot): 0

51 lines under test/bench/ci.

• src/serialize.h +126/-84
• src/bench/checkblock.cpp +31/-6
• src/hash.h +23/-1
• src/streams.h +18/-0
• src/crypto/sha256.cpp +15/-0
• src/streams.cpp +13/-0
• src/test/streams_tests.cpp +3/-3
• src/test/fuzz/autofile.cpp +2/-2
• src/bench/rpc_blockchain.cpp +1/-1
• src/test/crypto_tests.cpp +1/-1
• src/crypto/sha256.h +1/-0

This PR optimizes serialization primitives and SizeComputer by introducing single-byte specializations and static span extents to eliminate memcpy calls and buffer overheads. The goal is to speed up block serialization and size calculations, with claimed initial improvements on synthetic microbenchmarks. However, reviewers pointed out that production block IBD already runs through buffered streams, questioning whether real-world workloads benefit from this added template complexity. The PR is currently in draft, has merge conflicts, and is waiting on the author to retest and rework locally.