#29491 [EXPERIMENTAL] Schnorr batch verification for blocks

ReviewabilityReviewsAgreementSize
#29491 fjahr [EXPERIMENTAL] Schnorr batch verification for blocksCI failing0Strong2,452 + 276 tests
  • Speed up block validation and initial sync by batch verifying Taproot Schnorr signatures
  • Reduce CPU overhead during block connection by verifying signature cache misses simultaneously
  • CI is failing, so verify whether the failure is related to this PR before doing a deep review
  • No review verdicts recorded
  • Substantive support after earlier performance and lock contention issues were resolved (Sjors, Eunovo)
  • Support highlighted speedups on Taproot blocks and batch tweak verification readiness (Sjors, Eunovo)
  • Open request to document how batch verification and signature collection works (Sjors)
  • 2,452 lines added or modified outside tests
  • 276 lines added or modified in tests
  • 172 lines removed in total
  • 56 files, 11 commits
Utilities (logging, arguments, libraries)#14 of 59
P2 · speedup
  • P2 because batch verification in libsecp256k1 delivers up to 2x or greater speedups for Schnorr signatures
  • Adds unmerged cryptographic primitives that require careful algorithmic verification
  • Ranking pass: Kept P2 for the validation speedup, but placed below shipped-primitive work since it rests on unmerged experimental upstream modules.
Ranking notes
Validation#22 of 53
P2 · speedup
  • P2 because batching cache misses significantly cuts CPU overhead during IBD and block connection
  • Provides the largest validation speedups on blocks with heavy Taproot usage
  • Ranking pass: Moved down: benefit is real but gated on unmerged upstream secp256k1 work and explicitly experimental
Ranking notes
Reviewability: ReadyStalePaused
Agreement: StrongPositivePositive w/ caveatsNeutralMildDisputedBlockedCrickets
Reviews: current code-review ACKs, then (+stale ACKs) and -NACKs; greener = more ACKs.
Size: lines added or modified outside tests, then in tests; greener = smaller.

Goal

Vendors an experimental batch verification module in libsecp256k1 and integrates Schnorr signature batch verification into block validation. In both single-threaded and parallel script check queues, signatures that miss the signature cache are collected into a batch and verified simultaneously via multi-scalar multiplication before a block is accepted.

Problem: Schnorr signatures in Taproot inputs are verified individually during block validation. Validating signatures one by one incurs full per-signature cryptographic verification overhead rather than leveraging the significant CPU speedup available from BIP 340 batch verification.

Evidence of importance:

  • secp256k1 batch benchmarks in src/secp256k1/doc/speedup-batch/bench_output.txt and speedup graphs show up to 2x or greater verification speedups when batching Schnorr signatures compared to individual verification.
  • PR description notes this is a draft implementation intended for 'conceptual discussion, and collaborative benchmarking' while secp256k1 batch verification is under upstream review.

Scope: The PR vendors unmerged upstream secp256k1 work (secp256k1 PRs #1134 and #1789) directly into the src/secp256k1 subtree, accounting for most of the diff size.

Risk of a mistake: high (consensus, validation). A flaw in batch scalar arithmetic or batch queue logic could incorrectly validate invalid signatures or reject valid blocks, causing consensus divergences or remote validation failures.

Changed since the previous assessment: Rebased the vendored secp256k1 batch verification module to reflect recent upstream API changes (such as void add functions and context resetting), added a functional test for invalid Taproot output keys, and updated fuzz testing for the checkqueue.

What the assessment lacked:

  • rest_of_diff

Reviewability

Ready · CI failing

CI is red; check whether the failure is the PR's before a deep review.

Waiting on (from the thread read):

  • reviewer: review and benchmarks of the rebased branch 2026-10-30
  • reviewer: review the latest rebase and secp256k1 batch module API updates 2026-10-30

Agreement

Strong

substantive support, no open objection (Sjors, Eunovo)

Thread notes: Reread the discussion under schema 2: captured Sjors's initial question and suggestions regarding batching and configuration, and re-evaluated clears_with on performance defects. The PR remains an experimental draft awaiting reviewer benchmarks on the latest rebase. / Re-read discussion under schema 2: added Sjors' question and suggestions from c:1968590438 and c:1969001095; reclassified Eunovo's rc:2148806441 as a question; recorded latest rebase announcements by the author.

Objections, suggestions, and questions:

ReviewerTypeHarm, request, or questionStatusMust beAuthor repliedQuote
Sjors (member)question, approachhow batches are collected up to max_batch_size before calling batch.Verify()accepted · fjahr explained the batching mechanism and Sjors confirmed it clarified thingsansweredyes2024-02-28: 'Is the current PR already using batches? It's unclear to me by glancing over validation.cpp if and how it's collecting up to max_batch_size{106} signatures before calling batch.Verify().'
Status from: 2024-02-28: Sjors: 'That clarifies a lot, and should be documented :-)'
Sjors (member)suggestion, interfaceadd a -schnorrbatchverify startup argument so more people can try itanswered · fjahr agreed and considered adding a startup argument or compiler flagansweredyes2024-02-28: 'Maybe add a `-schnorrbatchverify` startup argument so more people can try it once it gets through review.'
Status from: 2024-02-28: fjahr: 'Yeah, I have been thinking that or maybe even a compiler argument'
Sjors (member)suggestion, maintenancedocument how batch verification and signature collection worksopenansweredno2024-02-28: 'That clarifies a lot, and should be documented :-)'
0xB10C (member)objection, approachbatch validation caused significant performance regressions during IBD compared to masterfixed · fix pushed · author implemented optimizations including batch per thread and verification outside critical sectionschangedyes2024-05-13: 'master is currently faster... after taproot activation, batch validation is significantly slower than non-batch validation'
Status from: 2025-05-18: fjahr: 'I have implemented this and see a performance boost from it for the case with 15 worker threads'
andrewtoth (member)objection, approachholding a shared batch mutex during addition and verifying on the main thread negated multithreading benefitsfixed · fix pushed · author implemented per-thread batches verified at the end of block checkschangedyes2024-12-07: 'm_batch_mutex is held for the entirety of adding, which will cause lock contention. Verify is then a blocking call on the main thread, which negates the multithreading.'
Status from: 2025-03-23: fjahr: 'I have now implemented it with a batch per thread using a flag set in complete as you suggested.'
Eunovo (member)objection, approachsharing a single batch across threads caused worker threads to block waiting for batch accessfixed · fix pushed · author implemented per-thread batcheschangedyes2025-03-17: 'All the threads still share this batch. That means we still have the same locking problem; threads will be stuck waiting for the batch to be free.'
Status from: 2025-03-23: fjahr: 'I have now implemented it with a batch per thread using a flag set in complete as you suggested.'
Eunovo (member)objection, approachperforming batch verification inside the critical section caused throughput to degrade as thread count increasedfixed · fix pushed · author moved batch verification outside the critical section and benchmarked speedupchangedyes2025-05-05: 'Doing batch-verification work within the critical section'
Status from: 2025-06-12: Eunovo: 'It seems like the major reason for performance drop was the batch-verification work being performed in the critical section.'
Eunovo (member)objection, correctnessdiscarding the return value of pubkey parsing risks ignoring invalid public keysfixed · fix pushed · author updated the code to check the return value in a later pushchangedyes2025-06-15: 'why discard the return value of secp256k1_xonly_pubkey_parse()?'
Status from: 2026-01-26: fjahr: 'Not discarding it anymore in the latest push.'

Support:

  • Sjors (member) (2024-02-28) [read: validation.cpp]: highlighted potential performance benefits on recent Taproot-heavy mainnet blocks and offered an assumeutxo snapshot to facilitate testing
  • Eunovo (member) (2025-03-16) [read: src/script/sigcache.h]: noted the architecture sets up well for future batch taproot tweak verification

Participants:

  • Sjors (member, support, 2 statements 2024-02-28 to 2024-02-28): supportive of testing against recent Taproot-heavy blocks and offered to provide an assumeutxo snapshot
  • 0xB10C (member, objection, 1 statements 2024-05-13 to 2024-05-13): benchmarked IBD showing batch validation was slower than master
  • willcl-ark (member, neutral, 1 statements 2024-05-13 to 2024-05-13): collaborated on benchmarks and noted lack of fallback mechanism in early WIP
  • andrewtoth (member, objection, 2 statements 2024-12-07 to 2025-03-20): objected to shared batch mutex contention and suggested per-thread batches
  • Eunovo (member, objection, 10 statements 2025-03-16 to 2025-06-15): pointed out shared batch lock contention, verification inside critical sections, and ignored pubkey parse return values

Files

335 lines under test/bench/ci.

  • src/secp256k1/src/modules/schnorrsig/batch_add_tests_impl.h +245/-0
  • src/secp256k1/src/modules/batch/main_impl.h +202/-0
  • src/secp256k1/src/tests.c +137/-35
  • src/checkqueue.h +125/-29
  • src/secp256k1/src/modules/schnorrsig/batch_add_impl.h +154/-0
  • src/secp256k1/examples/batch.c +152/-0
  • src/secp256k1/src/modules/extrakeys/batch_add_impl.h +147/-0
  • src/secp256k1/src/modules/extrakeys/bench_impl.h +138/-0
  • src/secp256k1/src/modules/extrakeys/batch_add_tests_impl.h +134/-0
  • src/secp256k1/src/modules/batch/tests_impl.h +126/-0
  • src/secp256k1/doc/speedup-batch/bench_output.txt +119/-0
  • src/test/key_tests.cpp +85/-18
  • test/functional/feature_taproot_invalid_output_key.py +93/-0
  • src/secp256k1/include/secp256k1_batch.h +88/-0
  • test/functional/feature_taproot.py +38/-37
  • src/secp256k1/src/ecmult_impl.h +54/-19
  • src/validation.cpp +50/-7
  • src/test/fuzz/batchverify.cpp +52/-0
  • src/secp256k1/src/modules/schnorrsig/tests_impl.h +47/-3
  • src/batchverify.h +49/-0
  • src/batchverify.cpp +48/-0
  • src/secp256k1/src/modules/schnorrsig/bench_impl.h +46/-0
  • src/secp256k1/include/secp256k1_tweak_check_batch.h +45/-0
  • src/secp256k1/doc/speedup-batch/plot.gp +41/-0
  • src/secp256k1/include/secp256k1_schnorrsig_batch.h +39/-0
  • src/secp256k1/src/bench.c +37/-2
  • src/secp256k1/.github/workflows/ci.yml +25/-13
  • src/secp256k1/doc/speedup-batch/Makefile +27/-0
  • src/validation.h +23/-4
  • src/script/sigcache.cpp +22/-0
  • src/script/sigcache.h +20/-0
  • src/secp256k1/doc/speedup-batch.md +17/-0
  • src/secp256k1/Makefile.am +15/-0
  • src/secp256k1/doc/speedup-batch/bench.sh +13/-0
  • src/secp256k1/src/CMakeLists.txt +11/-0
  • src/secp256k1/configure.ac +10/-0
  • src/secp256k1/src/secp256k1.c +10/-0
  • src/secp256k1/src/modules/extrakeys/Makefile.am.include +8/-0
  • src/test/txvalidationcache_tests.cpp +4/-4
  • src/secp256k1/src/modules/schnorrsig/Makefile.am.include +7/-0
  • src/secp256k1/examples/CMakeLists.txt +4/-0
  • src/script/script_error.h +3/-0
  • src/secp256k1/ci/ci.sh +2/-1
  • src/secp256k1/src/modules/batch/Makefile.am.include +3/-0
  • src/script/script_error.cpp +2/-0
  • src/secp256k1/CMakeLists.txt +2/-0
  • src/test/fuzz/checkqueue.cpp +2/-0
  • src/CMakeLists.txt +1/-0
  • src/kernel/CMakeLists.txt +1/-0
  • src/secp256k1/.gitignore +1/-0
  • src/secp256k1/README.md +1/-0
  • src/secp256k1/doc/speedup-batch/.gitignore +1/-0
  • src/test/fuzz/CMakeLists.txt +1/-0
  • test/functional/test_runner.py +1/-0
  • src/secp256k1/doc/speedup-batch/schnorrsig-speedup-batch.png +None/-None
  • src/secp256k1/doc/speedup-batch/tweakcheck-speedup-batch.png +None/-None

Card

The short text the ranking pass compares across PRs.

This pull request integrates BIP 340 Schnorr signature batch verification into block validation using an experimental secp256k1 batch module. Taproot Schnorr signatures are currently checked individually, missing the mathematical performance gains of batch verification during block connection and initial block download. Vendored benchmarks demonstrate substantial speedups over individual verification as batch sizes grow. The change depends conceptually on upstream secp256k1 PRs #1134 and #1789.

Data

dossier JSON · extract JSON · model ledger, generated 2026-11-01T01:16, confidence high, input hash 4851206d1ec7e5cf
thread read through 2026-10-30 (26 statements), head b7d2c11d4d
labels: CI failed · draft

Categories

Utilities (logging, arguments, libraries)

P2 · speedup (ranking pass: Kept P2 for the validation speedup, but placed below shipped-primitive work since it rests on unmerged experimental upstream modules.)

Introducing cryptographic batch verification primitives to libsecp256k1 delivers up to 2x or greater verification speedups for Schnorr signatures. The change requires careful review of cryptographic algorithms and subtree modifications.

Membership: Vendors an experimental batch verification module and tests into the src/secp256k1 subtree.

Factors: security/stability 0, bug 0, performance 2, user value 1, leverage 1

Validation

P2 · speedup (ranking pass: Moved down: benefit is real but gated on unmerged upstream secp256k1 work and explicitly experimental)

Batch verification provides substantial CPU performance gains during block validation and IBD for Taproot-heavy blocks. Caching misses are aggregated and verified via multi-scalar multiplication before a block is connected.

Membership: Modifies src/checkqueue.h and src/validation.cpp to batch verify Taproot Schnorr signatures during block connection.

Factors: security/stability 0, bug 0, performance 2, user value 1, leverage 1