#36121 fuzz: Corpus sharding in parallel runs
https://github.com/bitcoin/bitcoin/pull/36121 · · +70/-29 in 2 files, 2 commits · labels: Fuzzing · draft
Goal
- Parallelize individual fuzz targets across workers by splitting their corpora
- Prevent single long-running fuzz targets from bottlenecking multi-core test runs
This pull request updates the fuzz test runner to split target corpora into equal-sized shards according to the requested parallelism (`--par`). It also configures ramdisk on macOS and tmpfs container mounts in CI runner scripts to mitigate disk IO overhead during shard execution.
Problem: Currently, the fuzz test runner only parallelizes across distinct targets. When a single target is run or when a slow target dominates the run time, multi-core parallelism is underutilized, bottlenecking CI and local developer fuzzing.
Category: Test infrastructure (#20 of 45)
P3 · speedup
- P3 because it speeds up fuzz test runs by eliminating the bottleneck from the slowest target
- Breaks aggregated libFuzzer coverage metrics and adds target setup overhead
Corpus sharding significantly speeds up fuzz testing runs on multi-core machines, yielding 7 to 15 minute reductions on Windows, ASan, and MSan CI jobs, though with mixed performance on macOS due to storage IO.
Membership: Modifies test/fuzz/test_runner.py to shard fuzz test corpora across parallel jobs
Factors: security/stability 0, bug 0, performance 2, user value 0, leverage 1
Category: Build and CI (#43 of 55)
P3 · speedup
- P3 because it speeds up heavy CI fuzz jobs by 7 to 15 minutes using ramdisks and tmpfs
- Internal performance optimization for CI runs rather than a build reliability fix
Adds ramdisk and tmpfs container mounts in CI scripts to prevent disk IO contention during fuzz test runs.
Membership: Modifies ci/test/02_run_container.py to configure ramdisk on macOS and tmpfs mounts in CI containers
Factors: security/stability 0, bug 0, performance 1, user value 0, leverage 0
Reviewability: Ready
- Ready for review, CI passing and author addressed initial feedback
The code and CI modifications are complete and ready for review, despite draft status and ongoing performance evaluation.
Author status: active; last pushed CI tmpfs adjustments on 2026-09-01
Open concerns:
- Sharding prevents aggregate target coverage reporting in test output
- Targets with heavy per-input setup (such as utxo_total_supply) experience higher cumulative overhead when split across processes
- Author indicated they may close the pull request if platform-specific storage trade-offs become too complex
Resolved concerns:
- Temporary directory collisions between jobs resolved by using per-job shard directories
- macOS disk IO slowdown investigated and partially addressed via ramdisk mounts
Agreement: Strong
- Concept approval and verified speedups on MSan and Windows (jeanpablojp)
- Concern that sharding fragments libFuzzer output so coverage cannot be compared across runs (jeanpablojp)
- Concern about storage thrashing and target instantiation overhead on slower disks (jeanpablojp)
Concept ACK from jeanpablojp; concerns around per-target setup overhead and coverage loss were addressed or acknowledged
There is clear concept support from the sole reviewer, and objections regarding macOS runtime bottlenecks and shard directory management have been addressed with fixes and rationale.
- jeanpablojp: 'Concept ACK'
- maflcko: 'asan is 9 min faster (nice)... msan is 15 min faster... Windows is 7 min faster'
Objections:
| Reviewer | Kind | Harm | Status | Blocking | Author replied | Quote |
|---|---|---|---|---|---|---|
| jeanpablojp | maintenance | Fuzz coverage summaries are split across processes and cannot be aggregated per target to compare coverage across commits | resolved | no | yes | 2026-08-31: 'cov: becomes per process... you can't compare coverage across a change any more. Could the shards be folded back into one line per target?' Settled: 2026-09-01: maflcko replied 'I don't think it is possible to recover the coverage from the shards...' |
Support:
- jeanpablojp: Concept ACK on improving fuzz run parallelism
Participants: jeanpablojp (objection)
State derived from the lists: substantive support, no open objection (jeanpablojp) (model's own read: Positive)
Review verdicts (DrahtBot): 0
- Concept ACK: jeanpablojp
Files
99 lines under test/bench/ci.
- test/fuzz/test_runner.py +56/-29
- ci/test/02_run_container.py +14/-0
Card
PR #36121 enables corpus sharding within test/fuzz/test_runner.py to allow parallel execution of single fuzz targets and reduce the tail latency of slow targets. CI benchmark numbers show speedups of 7 to 15 minutes on ASan, MSan, and Windows jobs, alongside tmpfs adjustments in ci/test/02_run_container.py to reduce IO overhead. Feedback from jeanpablojp includes a Concept ACK with constructive feedback on target setup costs and coverage logging. The PR is reviewable and under active evaluation.