#33034 wallet: Store transactions in a separate sqlite table
https://github.com/bitcoin/bitcoin/pull/33034 · · +1055/-256 in 15 files, 27 commits · labels: Wallet, CI failed · draft
Goal
- Store transactions in a dedicated SQLite table instead of generic key-value records
- Allow deterministic in-order loading to simplify wallet logic like tracking spent outputs
This pull request migrates transaction persistence in SQLite wallets from generic key-value records to a dedicated 'transactions' relational table. Each transaction field is stored in its own column, allowing the wallet to load records ordered by position. It also introduces automatic schema upgrade and downgrade tracking using client feature flags.
Problem: The wallet treats SQLite as a simple key-value store, forcing complex composite serialization and resulting in arbitrary record loading order. This out-of-order load complicates wallet logic that relies on transaction sequences, such as tracking spent outputs.
Category: Wallet (#19 of 84)
P2 · unblocks #27865
- P2 because it directly unblocks tracking spent transaction outputs in #27865
- Provides deterministic in-order loading via SQL queries instead of arbitrary record loads
- Significantly improves maintainability by removing bespoke composite serialization
Refactoring SQLite wallet storage from a flat key-value store to dedicated relational tables is a major maintainability improvement. It resolves longstanding serialization headaches and provides deterministic in-order loading via SQL queries, directly unblocking PRs like #27865 that track spent transaction outputs.
Membership: Directly alters wallet persistence, SQLite schema, transaction serialization, and database loading in src/wallet/.
Factors: security/stability 1, bug 0, performance 1, user value 1, leverage 2
Reviewability: Stale: CI failing
CI is failing on the latest push, and the PR depends on unmerged PRs #32895 and #33033.
Author status: active
Agreement: Strong
- Strong concept support with no objections
- Concept approval for using relational tables over complex serialization workarounds (rkrux)
- Concept approval without stated reasons (w0xlt)
Strong concept support from rkrux citing cleaner serialization and relational DB benefits; no objections.
Two contributors have provided Concept ACKs, with rkrux explicitly validating the shift toward relational schema to avoid bespoke serialization workarounds.
- rkrux: Concept ACK noting agreement with using SQLite relational capabilities for transactions to avoid complex serialization
- w0xlt: Concept ACK
Review verdicts (DrahtBot): 0
Dependencies
Files
14 lines under test/bench/ci.
- src/wallet/sqlite.cpp +476/-195
- src/wallet/walletdb.cpp +224/-30
- src/wallet/sqlite.h +86/-15
- src/wallet/transaction.h +87/-0
- src/wallet/transaction.cpp +66/-0
- src/wallet/wallet.cpp +53/-4
- src/wallet/walletdb.h +18/-8
- src/wallet/walletutil.h +23/-0
- test/functional/wallet_backwards_compatibility.py +8/-2
- src/wallet/wallet.h +6/-0
- test/functional/wallet_createwallet.py +2/-2
- src/primitives/transaction_identifier.h +3/-0
- src/wallet/db.h +1/-0
- src/wallet/export.cpp +1/-0
- src/wallet/migrate.h +1/-0
Uncertainties
- Whether reviewer consensus will prefer migrating keys and metadata in the same transition or piecemeal across tables
Card
This PR begins shifting Bitcoin Core SQLite wallets away from flat key-value storage toward relational tables, starting with a dedicated transactions table. Storing transactions with explicit columns allows querying by wallet assignment order, simplifying downstream logic for transaction state and output tracking. Downstream work such as tracking no-longer-spendable TXOs in #27865 explicitly benefits from this structure. Review is currently blocked by failing CI runs and dependencies on #32895 and #33033, though initial concept feedback has been strongly supportive.