#36147 ci: add [free|open|net]BSD ci test jobs

ReviewabilityReviewsAgreementSize
#36147 willcl-ark ci: add [free|open|net]BSD ci test jobsNeeds rebase0Strong117 + 105 tests
  • Run CI tests on BSD platforms to catch platform-specific regressions before merge
  • Prevent recurring BSD breakage that currently requires reactive fixes after nightly runs
  • Review is worthwhile now as the rebase conflict with master is purely mechanical
  • Concept approval for running tests on at least one BSD platform (hebasto)
  • Concern that brittle nested VM tests could cause frequent false-positive CI failures is answered (maflcko)
  • 117 lines added or modified outside tests
  • 105 lines added or modified in tests
  • 141 lines removed in total
  • 14 files, 4 commits
Build and CI#23 of 56
P3 · test coverage
  • P3 because catching BSD regressions before merge prevents broken master builds and reactive fixes
  • The benefit is limited to BSD platform support rather than broader user-facing functionality
  • Ranking pass: Moved down from 13: scope still unsettled (likely FreeBSD only) and benefit is limited to BSD platform support
Ranking notes
Reviewability: ReadyStalePaused
Agreement: StrongPositivePositive w/ caveatsNeutralMildDisputedBlockedCrickets
Reviews: current code-review ACKs, then (+stale ACKs) and -NACKs; greener = more ACKs.
Size: lines added or modified outside tests, then in tests; greener = smaller.

Goal

Replaces the existing cross-compile CI jobs for FreeBSD, NetBSD, and OpenBSD with native compilation and test execution inside QEMU-based virtual machines using vmactions. Also includes small functional test framework adjustments to suppress FreeBSD lsof warnings and bound HTTPConnection socket timeouts.

Problem: Bitcoin Core cross-compiles for several BSD targets but does not execute tests against them in PR CI. Regressions on these platforms frequently slip into master and must be resolved reactively after nightly test failures.

Reviewability

Ready · Needs rebase

Conflicts with master; review the logic now, the rebase is mechanical.

Agreement

Strong

substantive support, no open objection (hebasto)

Thread notes: Initial record. maflcko cautioned that brittle BSD VM jobs could introduce high false-positive rates to CI, while hebasto offered a Concept ACK for including at least FreeBSD.

Objections, suggestions, and questions:

ReviewerTypeHarm, request, or questionStatusMust beAuthor repliedQuote
maflcko (member)objection, maintenancenested VM tests are brittle and will cause intermittent false positives on the main CI path, leading developers to ignore CI failuresanswered · reply on record; standing not assessed yet · v1, not yet re-checkedansweredyes2026-09-02: 'Those vm-in-a-vm tests are extremely brittle ... devs will just ignore all CI results because the false-positive rate is too high'

Support:

  • hebasto (member) (Concept ACK, 2026-09-03): having at least one CI job that runs tests on a *BSD platform

Participants:

  • hebasto (member, support, 2 statements 2026-09-02 to 2026-09-03): Concept ACK on running tests on at least one *BSD platform, proposing FreeBSD
  • maflcko (member, objection, 4 statements 2026-09-02 to 2026-09-17): raised concerns about nested VM test flakiness and high false-positive rates on CI, suggesting a trial or limiting scope to FreeBSD

Files

228 lines under test/bench/ci.

  • .github/workflows/ci.yml +117/-18
  • ci/test/00_setup_env_netbsd_cross.sh +0/-38
  • ci/test/00_setup_env_openbsd_cross.sh +0/-38
  • ci/test/00_setup_env_freebsd_cross.sh +0/-36
  • ci/test/00_setup_env_netbsd_native.sh +24/-0
  • ci/test/00_setup_env_openbsd_native.sh +24/-0
  • ci/test/03_test_script.sh +16/-6
  • ci/test/00_setup_env_freebsd_native.sh +21/-0
  • ci/README.md +12/-0
  • ci/test/02_run_container.py +3/-1
  • ci/test/00_setup_env.sh +2/-1
  • ci/test_run_all.sh +1/-1
  • test/functional/interface_rest.py +1/-1
  • test/functional/test_framework/netutil.py +1/-1

Card

The short text the ranking pass compares across PRs.

This PR replaces cross-compilation CI tasks for FreeBSD, NetBSD, and OpenBSD with VM-based native builds and functional test runs. It aims to eliminate a recurring cycle where BSD regressions pass PR review and are only caught by external nightly builds. Review is currently stale due to merge conflicts and open discussion regarding whether nested VM flakiness makes running all three BSDs on every PR too noisy. The author and reviewers have converged toward testing only FreeBSD if the PR proceeds.

Data

dossier JSON · extract JSON · model ledger, generated 2026-09-18T03:45, confidence high, input hash 3d8d7bc0eadc5398
thread read through 2026-09-17 (8 statements), head 31a2ac55c0
Discussion read under schema 1 rules: suggestions and questions may be missing, and an objection may combine several harms (schema history).
labels: Tests, Needs rebase · draft

Categories

Build and CI

P3 · test coverage (ranking pass: Moved down from 13: scope still unsettled (likely FreeBSD only) and benefit is limited to BSD platform support)

Expanding CI to run tests on BSD platforms helps catch platform-specific regressions before they reach master. The author notes that without PR test runs, BSD regressions frequently slip into master and require reactive fixes, though concerns about nested VM brittleness on main CI workflows keep this at P3.

Membership: Modifies .github/workflows/ci.yml and multiple ci/test/* scripts to replace cross-compile jobs with native compile and test execution in BSD VMs.

Factors: security/stability 0, bug 0, performance 0, user value 0, leverage 1