{ "number": 35068, "repo": "bitcoin/bitcoin", "url": "https://github.com/bitcoin/bitcoin/pull/35068", "title": "depends: fix cmake package discovery for depends-built packages", "author": "ryanofsky", "author_association": "CONTRIBUTOR", "created_at": "2026-04-13T12:00:37Z", "updated_at": "2026-08-10T19:22:36Z", "age_days": 157, "draft": false, "labels": [ "Build system" ], "milestone": null, "base": "master", "head_sha": "9a7f122d07005021252ca8f3c272cae740fefb26", "head_ref": "pr/depfind", "head_repo": "ryanofsky/bitcoin", "head_history": [], "additions": 1, "deletions": 1, "changed_files": 1, "commit_count": 1, "size_bucket": "S", "mergeable_state": "unknown", "bot": { "drahtbot": { "present": true, "reviews": {}, "conflicts": [] } }, "acks_parsed": {}, "acks_tally": { "ack": 0, "stale_ack": 0, "concept_ack": 0, "approach_ack": 0, "nack": 0, "concept_nack": 0, "approach_nack": 0 }, "reviews": { "approved": 0, "changes_requested": 0, "distinct_reviewers": [ "hebasto", "purpleKarrot", "sedited" ] }, "signals": { "needs_rebase": false, "ci_failed": false, "mergeable_state": "unknown", "last_author_activity": "2026-04-15T13:11:03Z", "last_reviewer_activity": "2026-08-10T19:22:31Z", "last_reviewer": "sedited", "author_silent_days": 155, "waiting_on_author_days": 37, "days_since_update": 37 }, "refs": { "mentioned": [], "depends_on": [], "fixes": [], "linked_issues": [], "references": [], "conflicts": [] }, "stack": { "shares_commits_with": [], "based_on": [], "base_for": [] }, "review_paths": [ "depends/funcs.mk" ], "body": "When building cmake-based depends packages (e.g. native_libmultiprocess), cmake needs to find dependencies that were previously built and installed into the depends prefix. Since 603fd6a2e70 (\"depends: add MULTIPROCESS depends option\"), funcs.mk has set CMAKE_MODULE_PATH in the environment to point cmake at the depends prefix. This never worked for two reasons:\n\n1. CMake does not read CMAKE_MODULE_PATH from the environment; it only honors it when set via -D flags or set() in CMakeLists.\n\n2. CMAKE_MODULE_PATH only affects module-mode find_package() (Find*.cmake files), but CapnProto and most modern cmake packages use config mode (NO_MODULE / *Config.cmake files), which ignores CMAKE_MODULE_PATH entirely and instead uses CMAKE_PREFIX_PATH.\n\nCMake does read CMAKE_PREFIX_PATH from the environment, and it is the correct variable for config-mode package discovery. Replace the broken CMAKE_MODULE_PATH assignment with CMAKE_PREFIX_PATH prepended to any existing value (mirroring how PATH is handled), pointing at the type-appropriate depends prefix.\n\nThis fixes a build failure in native_libmultiprocess where cmake would find the system (nix) CapnProto 1.2.0 instead of the depends-built CapnProto 1.4.0, resulting in a compile-time version mismatch error:\n #error \"Version mismatch between generated code and library headers.\"", "commits": [ { "sha": "9a7f122d07005021252ca8f3c272cae740fefb26", "date": "2026-04-10T21:25:36Z", "message": "depends: fix cmake package discovery for depends-built packages\n\nWhen building cmake-based depends packages (e.g. native_libmultiprocess),\ncmake needs to find dependencies that were previously built and installed\ninto the depends prefix. Since 603fd6a2e70 (\"depends: add MULTIPROCESS\ndepends option\"), funcs.mk has set CMAKE_MODULE_PATH in the environment\nto point cmake at the depends prefix. This never worked for two reasons:\n\n1. CMake does not read CMAKE_MODULE_PATH from the environment; it only\n honors it when set via -D flags or set() in CMakeLists.\n\n2. CMAKE_MODULE_PATH only affects module-mode find_package() (Find*.cmake\n files), but CapnProto and most modern cmake packages use config mode\n (NO_MODULE / *Config.cmake files), which ignores CMAKE_MODULE_PATH\n entirely and instead uses CMAKE_PREFIX_PATH.\n\nCMake does read CMAKE_PREFIX_PATH from the environment, and it is the\ncorrect variable for config-mode package discovery. Replace the broken\nCMAKE_MODULE_PATH assignment with CMAKE_PREFIX_PATH prepended to any\nexisting value (mirroring how PATH is handled), pointing at the\ntype-appropriate depends prefix.\n\nThis fixes a build failure in native_libmultiprocess where cmake would\nfind the system (nix) CapnProto 1.2.0 instead of the depends-built\nCapnProto 1.4.0, resulting in a compile-time version mismatch error:\n #error \"Version mismatch between generated code and library headers.\"\n\nCo-Authored-By: Claude Sonnet 4.6 " } ], "timeline": [ { "t": "2026-04-13T13:00:59Z", "kind": "review_comment", "who": "hebasto", "assoc": "MEMBER", "path": "depends/funcs.mk", "commit": "9a7f122d07005021252ca8f3c272cae740fefb26", "in_reply_to": null, "text": "Keeping the `CMAKE_PREFIX_PATH` value potentially set externally seems to me to undermine the purpose of the depends build subsystem, no?" }, { "t": "2026-04-13T13:04:12Z", "kind": "comment", "who": "hebasto", "assoc": "MEMBER", "text": "Why not explicitly set `CMAKE_PREFIX_PATH` only where it's needed?\n\nFor example:https://github.com/bitcoin/bitcoin/blob/976985eccd546a95e38973b854ccc6589e8afc74/depends/packages/qt.mk#L172" }, { "t": "2026-04-13T13:12:08Z", "kind": "comment", "who": "hebasto", "assoc": "MEMBER", "text": "[quoted text omitted]\n\nWhy doesn't setting `CMAKE_INSTALL_PREFIX` help?\nhttps://github.com/bitcoin/bitcoin/blob/976985eccd546a95e38973b854ccc6589e8afc74/depends/funcs.mk#L221\n\nAccording to the [documentation](https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html), it gets included in the search paths by default." }, { "t": "2026-04-13T13:45:10Z", "kind": "review_comment", "who": "ryanofsky", "assoc": "CONTRIBUTOR", "path": "depends/funcs.mk", "commit": "9a7f122d07005021252ca8f3c272cae740fefb26", "in_reply_to": 3073140573, "text": "re: https://github.com/bitcoin/bitcoin/pull/35068#discussion_r3073140573\n\n[quoted text omitted]\nI don't think it should be a goal of the depends system to *individually* reset external environment variables that could affect the build in every build command. IMO it makes sense for the depends system to just be responsible for building dependencies, and for the packaging system (guix, or shell.nix, or external build scripts) to be responsible for setting up the build environment.\n\nSo this PR is being conservative and just putting depends packages ahead of other packages, not trying to change the build model or break things that are currently working.\n\nConversely, if we do want depends to be responsible for clearing environment variables, that would be reasonable, but it I think it would make more sense to implement it by either (1) using `env -i` to start from a clean environment and whitelisting needed variables like PATH or (2) compiling a list of variables used by cmake, autotool, pkgconfig and blocking them all in one place. But blocking individual variables in individual build commands seems like a recipe for confusion and bugs and does not seem like a good direction to go." }, { "t": "2026-04-13T14:27:31Z", "kind": "review", "who": "ryanofsky", "assoc": "CONTRIBUTOR", "state": "COMMENTED", "commit": "9a7f122d07005021252ca8f3c272cae740fefb26", "text": "Thanks for the review! Re: the suggestions to use [-DCMAKE_PREFIX_PATH](https://github.com/bitcoin/bitcoin/pull/35068#issuecomment-4236580116)\n or [-DCMAKE_INSTALL_PREFIX](https://github.com/bitcoin/bitcoin/pull/35068#issuecomment-4236629103), I do not think CMAKE_INSTALL_PREFIX is the right way to control the search paths CMake uses to find dependencies, because its primary purpose is to set the destination path where the package will be installed. It could make sense to set that variable for other reasons, but it would not address the underlying issue here. (For example, the bug would reappear if the depends system ever gained stronger separation between packages and no longer placed them all under the same prefix.)\n\nSetting -DCMAKE_PREFIX_PATH would be more reasonable than -DCMAKE_INSTALL_PREFIX, since it does affect dependency search, but I still think the better fix here is to set environment variables correctly (PATH, CMAKE_PREFIX_PATH, etc) in one place, instead of relying hardcoded on CMakeCache.txt settings to indirectly influence search behavior. Setting environment variables is preferable to hardcoding cache entries, because it makes fewer assumptions about how cmake projects manage the cache, and it ensures the configuration is inherited by subprocesses. Cache variables are a good way of controlling builds, but environment variables are a better way of providing information about build environments IMO." }, { "t": "2026-04-13T14:44:00Z", "kind": "comment", "who": "hebasto", "assoc": "MEMBER", "text": "[quoted text omitted]\n\nIn that comment, I'm not debating which approach is more reasonable. My point is that if the current code setting `CMAKE_INSTALL_PREFIX` doesn't work for you, but this branch, which additionally sets `CMAKE_PREFIX_PATH`, does, it might point to a deeper issue.\n\nBy the way, is the original issue Nix-specific? If not, could you provide the exact steps to reproduce it?" }, { "t": "2026-04-13T23:29:10Z", "kind": "comment", "who": "ryanofsky", "assoc": "CONTRIBUTOR", "text": "[quoted text omitted]\n\nOh, thanks. I don't see how it could point to deeper issues other than cmake's documentation being a little confusing, or nix overriding some of cmake's defaults, and we already know both things happen.\n\nSetting CMAKE_INSTALL_PREFIX isn't a reliable way to tell cmake where to find dependencies or how to prioritize different versions of the same dependency. If setting CMAKE_INSTALL_PREFIX happens to have positive side effects on some systems, that's great, but the point of CMAKE_INSTALL_PREFIX is to control where packages are installed not where dependencies are found.\n\nIf you think there could be a deeper issue here, could explain what type of issue you think it might be?\n\n[quoted text omitted]\nI think the main bug this PR is fixing is that the `CMAKE_MODULE_PATH` value depends currently specifies is ignored, and this bug is not nix-specific.\n\nI happened to encounter the problem by running `make -C depends` using a `shell.nix` with `capnproto` in its `buildInputs`. This caused nix to install capnproto 1.2 and libmultiprocess to use the capnproto 1.2 package and code generator. But at compile time the `-I/home/bitcoin/depends/x86_64-pc-linux-gnu/native/include` include path [added by depends](https://github.com/bitcoin/bitcoin/blob/976985eccd546a95e38973b854ccc6589e8afc74/depends/funcs.mk#L21) would cause capnproto 1.4 library headers to be preferred over capnproto 1.2 headers, leading to the error \"Version mismatch between generated code and library headers.\" (EDIT: fixed original description of the version mismatch which had some details wrong)" }, { "t": "2026-04-14T10:28:39Z", "kind": "comment", "who": "hebasto", "assoc": "MEMBER", "text": "cc @purpleKarrot" }, { "t": "2026-04-15T12:40:28Z", "kind": "comment", "who": "purpleKarrot", "assoc": "CONTRIBUTOR", "text": "There are three ways to approach this:\n\n## 1. Install all projects to a common prefix\n\nIn the simplest approach, you *configure*, *build*, (optionally *test*), and *install* all packages to a single install prefix. And you want to find dependencies in that same prefix. Hence, you would set **both** `CMAKE_INSTALL_PREFIX` and `CMAKE_PREFIX_PATH` to the **same** value for all projects. The former controls where to *put* things, the latter controls where to *search* things.\n\n## 2. Install each project to a dedicated prefix\n\nThis approach gives you more control which package may access which other package. You would set `CMAKE_INSTALL_PREFIX` differently for each project and you would explicitly set `_DIR` for each dependency.\n\n## 3. Consume projects from their build tree\n\nIn this approach, you don't perform the *install* steps, so setting `CMAKE_INSTALL_PREFIX` is superfluous. You would still set `_DIR` for each dependency.\n\n## Remarks\n\nApproaches 1 and 2 require that all projects properly install their targets using [`install(EXPORT)`](https://cmake.org/cmake/help/latest/command/install.html#export).\nApproach 3 requires that projects export their targets using [`export(EXPORT)`](https://cmake.org/cmake/help/latest/command/export.html#export).\n\nIt is good practice to use both `install(EXPORT)` and `export(EXPORT)` consistently in all CMake projects, so that all three approaches described above (plus `FetchContent`) work transparently and the downstream users may consume the project however they prefer. In practice, it is really rare to see a project that does it properly, because, well ...\n\n[quoted text omitted]\nAll of this is independent to CMake's modules. Hence, `CMAKE_MODULE_PATH` should not even be mentioned in the description." }, { "t": "2026-04-15T13:11:03Z", "kind": "comment", "who": "ryanofsky", "assoc": "CONTRIBUTOR", "text": "[quoted text omitted]\n\nThanks, CMAKE_MODULE_PATH is mentioned in the description because 9a7f122d07005021252ca8f3c272cae740fefb26 is dropping a broken CMAKE_MODULE_PATH assignment in the code." }, { "t": "2026-04-15T14:03:53Z", "kind": "review_comment", "who": "purpleKarrot", "assoc": "CONTRIBUTOR", "path": "depends/funcs.mk", "commit": "9a7f122d07005021252ca8f3c272cae740fefb26", "in_reply_to": 3073140573, "text": "What about `PKG_CONFIG_LIBDIR` and `PKG_CONFIG_PATH` then?\n\nI think that\n\n```diff\n-$(1)_config_env+=CMAKE_MODULE_PATH=$($($(1)_type)_prefix)/lib/cmake\n+$(1)_config_env+=CMAKE_PREFIX_PATH=$($($(1)_type)_prefix)\n```\n\nIs the proper fix for this issue. It may then be discussed separately whether the original env vars of `PKG_CONFIG_LIBDIR`, `PKG_CONFIG_PATH`, and `CMAKE_PREFIX_PATH` should be preserved." }, { "t": "2026-04-15T14:21:57Z", "kind": "comment", "who": "purpleKarrot", "assoc": "CONTRIBUTOR", "text": "Note also that support for [`CMAKE_INSTALL_PREFIX`](https://cmake.org/cmake/help/latest/envvar/CMAKE_INSTALL_PREFIX.html) as an **environment variable** requires CMake version 3.29.\nOur minimum supported version is still 3.22, so you should set the cmake variable via `-D`." }, { "t": "2026-08-10T19:22:31Z", "kind": "comment", "who": "sedited", "assoc": "CONTRIBUTOR", "text": "What is the status here? There seems to be some unaddressed review." } ], "labels_log": [ { "t": "2026-04-13T12:00:41Z", "action": "labeled", "label": "Build system", "who": "DrahtBot" } ], "state_log": [], "text_chars": 10885, "text_tokens_estimate": 2721, "changed_paths": [ "depends/funcs.mk" ], "files": [ { "path": "depends/funcs.mk", "add": 1, "del": 1 } ], "test_lines": 0, "git": { "head": "9a7f122d07005021252ca8f3c272cae740fefb26", "head_matches_backup": true, "base": "58dccd27e11de68f810145fc30631a2c446f3bf5", "commits": [ { "sha": "9a7f122d07", "subject": "depends: fix cmake package discovery for depends-built packages", "files": 1, "add": 1, "del": 1 } ], "patch_truncated": false }, "input_hash": "46abe6f733d3fbf6", "extracted_at": "2026-09-17T16:15:31+00:00" }