#36230 wallet: Improve `HasWalletDescriptor` performance and other canonical descriptor string followups
https://github.com/bitcoin/bitcoin/pull/36230 · · +86/-30 in 9 files, 7 commits · labels: Wallet, Needs Backport (32.x)
Goal
- Fix a noticeable lookup performance regression when handling complex descriptors like Miniscript
- Avoid repeatedly allocating and computing canonical descriptor strings during wallet checks
Caches the SHA-256 hash of the canonical descriptor string on `WalletDescriptor` to optimize `HasWalletDescriptor` checks and avoid recurring string allocations. It also removes `WalletDescriptor`'s default constructor in favor of explicit initialization, documents descriptor update behavior, and updates backwards compatibility testing data.
Problem: Recomputing the full canonical descriptor string during every `HasWalletDescriptor` call caused a noticeable performance regression when handling complex descriptors (such as Miniscript), cutting wallet descriptor lookup speed in half.
Category: Wallet (#6 of 84)
P2 · speedup
- P2 because it fixes a significant performance regression in descriptor comparison checks
- Halves descriptor lookup time for wallets holding complex descriptors like Miniscript
- Prevents the performance regression from persisting into release
P2 because it fixes a significant performance regression in descriptor comparison where repeated string generation slowed down wallet checks (confirmed by rkrux noting a 50% improvement in `wallet_miniscript.py`). Backporting to 32.x is requested, preventing the regression from persisting into release.
Membership: Modifies WalletDescriptor and DescriptorScriptPubKeyMan in src/wallet/ to cache canonical descriptor hashes and optimize descriptor lookup routines.
Factors: security/stability 0, bug 1, performance 2, user value 1, leverage 1
Reviewability: Ready
- Ready for review, with passing CI and prompt author updates
CI is clean and the author actively addresses review feedback within the same day.
Author status: active
Open concerns:
- Whether descriptor computation should be strictly lazy on first lookup rather than computed at construction
- Protecting `m_wallet_descriptor.descriptor` with `const` to prevent mutating the underlying descriptor while cached hashes exist
Resolved concerns:
- Removed the uninitialized default constructor in favor of RAII
- Renamed `Equals` to `IsCanonicallyEquivalent` to clarify that only canonical representations are compared
- Fixed backwards compatibility CI failures when adding version 31.1
Agreement: Positive
- Broad support for caching to eliminate repeated descriptor generation
- Verified by testing a 50% speedup on Miniscript wallet tests (rkrux)
- Verified test runtime dropped substantially on Alpine (hebasto)
- Suggested making cached descriptors const to prevent post-construction mutations (polespinasa, davidgumberg)
Positive; tested ACK from rkrux confirming a 50% speedup, with minor follow-up design suggestions from polespinasa and davidgumberg.
Reviewers verified the benchmark improvements and agreed with the caching concept while discussing implementation details regarding const-correctness and lazy evaluation.
- rkrux gave tested ACK reporting 50% improvement in wallet test execution time
- hebasto confirmed test runner execution time dropped substantially on Alpine
- polespinasa and davidgumberg suggested protecting descriptors against post-construction mutation
Review verdicts (DrahtBot): 0 (+1)
- Stale ACK: rkrux
- Concept ACK: polespinasa
Files
37 lines under test/bench/ci.
- src/script/descriptor.cpp +22/-4
- test/functional/wallet_backwards_compatibility.py +10/-14
- src/wallet/walletutil.h +21/-2
- src/wallet/walletutil.cpp +16/-1
- test/get_previous_releases.py +9/-0
- src/wallet/walletdb.cpp +4/-4
- src/wallet/test/wallet_tests.cpp +2/-2
- src/wallet/scriptpubkeyman.cpp +1/-2
- src/script/descriptor.h +1/-1
Card
This PR optimizes wallet descriptor comparisons by computing and caching a SHA-256 hash of the canonical descriptor string on WalletDescriptor rather than rebuilding strings during every HasWalletDescriptor check. This eliminates a 50% performance slowdown in descriptor comparison workloads and addresses documentation and string compatibility followups from earlier descriptor work. The change is marked for backport to 32.x, has positive review with a tested ACK, and has minor open discussions regarding lazy evaluation and const-correctness.