#29409 multiprocess: Add capnp wrapper for Chain interface
https://github.com/bitcoin/bitcoin/pull/29409 · · +2190/-429 in 71 files, 7 commits · labels: IPC
Goal
- Allow wallets and external clients to run as separate processes from the node
- Enable out-of-process clients to query chainstate and receive asynchronous chain notifications
This pull request defines Cap'n Proto schemas and C++ proxy wrappers for the `interfaces::Chain` and `interfaces::Handler` interfaces. It squashes upstream updates to the `libmultiprocess` subtree (v14) and integrates serialization helpers for Bitcoin Core types. This allows processes connected via UNIX sockets or socket pairs to access chain queries, mempool state, and notifications.
Problem: Currently, running the Bitcoin Core wallet or other services out-of-process requires an IPC communication channel for chain queries and notifications. Without a serialized Chain interface, the process separation project (#10102) cannot proceed and external consumers must rely on polling JSON-RPC.
Category: IPC / multiprocess (#2 of 20)
P2 · new feature
- P2 because it is a major prerequisite to run the wallet in a separate process
- Directly unblocks multiple downstream multiprocess PRs and external integrations
P2 because exposing `interfaces::Chain` over Cap'n Proto IPC is a critical foundational component for process separation, directly unblocking out-of-process wallet support (#10102) and enabling external clients like BDK and Electrs to interact with node chainstate over a structured socket.
Membership: Adds Cap'n Proto schemas and wrappers for the Chain interface and updates the libmultiprocess subtree.
Factors: security/stability 1, bug 0, performance 1, user value 2, leverage 3
Reviewability: Ready
- Ready for review with passing CI and no pending structural changes
The branch merges cleanly, CI passes, and the changes are stable enough that multiple contributors are running integration tests against them.
Author status: Active, rebasing regularly to track master and responding to feedback.
Open concerns:
- Node asserts or segfaults when non-C++ IPC clients supply out-of-bounds block heights or omit required parameters like transactions (zaidmstrr, willcl-ark).
- Unclean client disconnections during shutdown can cause proxy termination crashes, tracked in libmultiprocess#219 (sedited, darosior).
Resolved concerns:
- Missing struct fields such as `best_height` and mismatched integer/enum widths across C++ and capnp declarations (l0rinc).
- Subtree compile failures and lint errors during rebase updates (maflcko, zaidmstrr, pseudoramdom).
Agreement: Mild
- Strong support with testing against external client prototypes (darosior, sedited)
- Concept approval for enabling downstream client integration (willcl-ark, josibake)
- Questioned handling of invalid parameters from non-C++ clients (zaidmstrr, willcl-ark)
- Ongoing architectural discussion on raw versus specialized IPC interfaces (Sjors)
Strong concept and testing support, but nonblocking concerns remain open regarding crash safety when clients send null or invalid inputs.
Multiple reviewers have ACKed and developed functional downstream prototypes against the interface. Nonblocking objections from willcl-ark and zaidmstrr note that omitting arguments or requesting blocks past the tip triggers assertions or null pointer dereferences on the node side.
- darosior (2024-12-09): 'tested ACK 395d5eed... exercised getHeight, getBlockHash, hasBlocks... and ChainNotifications'
- willcl-ark (2026-05-20): 'Concept ACK. Exposing interfaces::Chain over IPC seems very useful...'
- zaidmstrr (2026-09-09): 'Assertion `chainman().ActiveChain()[height]` failed. Aborted (core dumped)'
Objections:
| Reviewer | Kind | Harm | Status | Blocking | Author replied | Quote |
|---|---|---|---|---|---|---|
| zaidmstrr | correctness | node aborts with an assertion failure if an IPC client requests getBlockHash for a height exceeding the active chain tip | open | no | no | 2026-09-09: 'when we pass the height, which is greater than the current tip, then the node crashes and exits with the error... Assertion `chainman().ActiveChain()[height]` failed.' |
| willcl-ark | safety | node segfaults if a non-C++ client passes null transaction data or an unhandled enum value over IPC | open | no | yes | 2026-05-20: 'Also what happens here is tx is a null pointer (allowed by capnp spec AFAIU)? I think we will immediately try to deference it in BroadcastTransaction and segfault.' Settled: 2026-05-20: 'In general it's not a realistic goal for this PR to ensure that it's safe to call all Chain methods with all possible arguments and ensure that the node will not crash, but it should be good to do where feasible.' |
| l0rinc | correctness | capnp definitions had missing struct fields and signedness/width discrepancies with C++ declarations | resolved | yes | yes | 2025-11-25: 'It seems to me the capnp wrapper needs some updates (missing field, 32/64 bit params, param rename, bool vs uint64, enums as unsigned, optional mapping, unused declarations, comments, etc).' Settled: 2025-12-12: author addressed all discrepancies in push 89cf624fe8 and l0rinc did not object further |
Support:
- darosior: verified multiple scenarios on regtest and signet using a custom Rust BDK wallet client
- sedited: reviewed code and tested against Rust client integrations
- willcl-ark: tested against a node dashboard and considers IPC the ideal programmatic boundary for Bitcoin Core
- Sjors: developed Rust bindings and demonstrated an Electrs integration over IPC
- josibake: Concept ACK [not substantive]
- cbergqvist: Surface-level ACK [not substantive]
- ariard: finds socket-based IPC very useful for separating node and wallet onto different hosts
Participants: cbergqvist (support), ariard (support), sedited (support), josibake (support), darosior (support), Big621 (neutral), pseudoramdom (question), zaidmstrr (objection), maflcko (neutral), l0rinc (objection), Sjors (support), xyzconstant (question), willcl-ark (objection)
State derived from the lists: nonblocking objection open (zaidmstrr, willcl-ark)
Review verdicts (DrahtBot): 0 (+2)
- Stale ACK: cbergqvist, sedited
- Concept ACK: josibake, darosior, willcl-ark
Dependencies
Files
0 lines under test/bench/ci.
- src/ipc/libmultiprocess/src/mp/util.cpp +216/-31
- src/ipc/libmultiprocess/test/mp/test/connect_tests.cpp +212/-0
- src/ipc/capnp/chain.cpp +208/-0
- src/ipc/libmultiprocess/test/mp/test/test.cpp +190/-6
- src/ipc/capnp/chain.capnp +189/-0
- src/ipc/capnp/common-types.h +168/-2
- src/ipc/libmultiprocess/src/mp/proxy.cpp +94/-35
- src/ipc/libmultiprocess/test/mp/test/listen_tests.cpp +47/-77
- src/ipc/capnp/chain-types.h +107/-0
- src/ipc/libmultiprocess/include/mp/proxy-io.h +74/-28
- src/ipc/libmultiprocess/test/mp/test/unixlistener.h +84/-0
- src/ipc/libmultiprocess/src/mp/gen.cpp +39/-41
- src/ipc/libmultiprocess/include/mp/util.h +63/-15
- src/ipc/capnp/common.capnp +58/-0
- src/ipc/libmultiprocess/include/mp/proxy-types.h +31/-12
- src/ipc/libmultiprocess/cmake/compat_config.cmake +0/-42
- src/ipc/libmultiprocess/include/mp/type-number.h +24/-15
- src/ipc/libmultiprocess/test/mp/test/spawn_tests.cpp +28/-5
- src/ipc/libmultiprocess/test/mp/test/foo.h +31/-1
- src/ipc/libmultiprocess/include/mp/type-chrono.h +25/-2
- src/ipc/libmultiprocess/test/mp/test/common.h +27/-0
- src/ipc/libmultiprocess/.github/workflows/ci.yml +15/-9
- src/ipc/libmultiprocess/CMakeLists.txt +17/-7
- src/ipc/libmultiprocess/include/mp/type-struct.h +12/-12
- src/ipc/libmultiprocess/include/mp/type-interface.h +10/-10
- src/ipc/libmultiprocess/ci/scripts/ci.sh +18/-0
- src/ipc/libmultiprocess/include/mp/type-context.h +15/-3
- src/ipc/libmultiprocess/shell.nix +12/-6
- src/ipc/capnp/handler.capnp +17/-0
- CMakeLists.txt +14/-0
- src/ipc/libmultiprocess/include/mp/proxy.h +8/-6
- src/ipc/libmultiprocess/example/calculator.cpp +4/-9
- src/ipc/libmultiprocess/example/printer.cpp +4/-9
- src/ipc/libmultiprocess/example/example.cpp +7/-5
- src/ipc/libmultiprocess/include/mp/type-char.h +9/-3
- src/ipc/libmultiprocess/test/mp/test/foo.capnp +12/-0
- src/ipc/capnp/handler-types.h +10/-0
- src/ipc/libmultiprocess/doc/versions.md +8/-2
- src/interfaces/chain.h +5/-4
- src/ipc/libmultiprocess/ci/configs/netbsd.bash +1/-7
- src/ipc/libmultiprocess/cmake/pthread_checks.cmake +8/-0
- src/ipc/libmultiprocess/include/mp/type-threadmap.h +4/-4
- src/ipc/libmultiprocess/ci/configs/newdeps.bash +6/-0
- src/ipc/libmultiprocess/doc/usage.md +6/-0
- src/ipc/libmultiprocess/test/mp/test/foo-types.h +6/-0
- src/node/interfaces.cpp +3/-3
- src/ipc/CMakeLists.txt +5/-0
- src/ipc/libmultiprocess/ci/configs/olddeps.bash +4/-1
- src/ipc/libmultiprocess/include/mp/type-data.h +2/-3
- src/ipc/libmultiprocess/ci/configs/sanitize.bash +2/-2
- src/ipc/libmultiprocess/doc/design.md +2/-2
- src/ipc/libmultiprocess/include/mp/type-string.h +3/-1
- src/ipc/capnp/init.capnp +3/-0
- src/ipc/libmultiprocess/.github/workflows/bitcoin-core-ci.yml +3/-0
- src/ipc/libmultiprocess/ci/configs/default.bash +1/-1
- src/ipc/libmultiprocess/ci/configs/freebsd.bash +1/-1
- src/ipc/libmultiprocess/ci/configs/llvm.bash +1/-1
- src/ipc/libmultiprocess/ci/configs/macos.bash +1/-1
- src/ipc/libmultiprocess/ci/configs/openbsd.bash +1/-1
- src/ipc/libmultiprocess/cmake/TargetCapnpSources.cmake +1/-1
- src/ipc/libmultiprocess/doc/install.md +1/-1
- src/ipc/libmultiprocess/include/mp/type-function.h +1/-1
- src/ipc/libmultiprocess/include/mp/type-map.h +2/-0
- src/ipc/libmultiprocess/include/mp/version.h +1/-1
- src/policy/fees/block_policy_estimator.h +2/-0
- src/rpc/request.h +2/-0
- src/ipc/capnp/init-types.h +1/-0
- src/ipc/libmultiprocess/.clang-tidy +1/-0
- src/ipc/libmultiprocess/ci/README.md +1/-0
- src/ipc/libmultiprocess/include/mp/config.h.in +1/-0
- src/ipc/libmultiprocess/test/CMakeLists.txt +1/-0
Card
PR #29409 provides Cap'n Proto schema definitions and C++ wrappers for the `interfaces::Chain` interface, allowing external processes to query chainstate and receive block notifications over IPC sockets. It is a prerequisite for running the wallet out-of-process in #10102 and has already been validated against independent external consumers such as BDK Rust wallets and Electrs. Review sentiment is broadly favorable with tested ACKs, though open nonblocking discussions highlight the need for safer input validation against null arguments and out-of-range heights from foreign language clients. The PR is rebased, passing CI, and ready for further review.