#31349 ci: detect outbound internet traffic generated while running tests

ReviewabilityReviewsAgreementSize
#31349 vasild ci: detect outbound internet traffic generated while running testsAuthor silent 136d1 (+2)Mild0 + 77 tests
  • Prevent tests in CI from making unexpected outbound internet connections or leaking DNS queries
  • Keep test runs hermetic to avoid leaking environment data or causing flakiness
  • Author has been inactive for 98 days
  • Review may wait on the author to address open feedback
  • Broad concept approval with multiple tested ACKs for blocking network leaks (0xB10C, Sjors, fjahr)
  • Code review approval on the latest revision (ryanofsky)
  • Unaddressed objection: packet direction flag is a no-op when reading capture files (fjahr)
  • Thinks the direction flag bug is harmless and would be good to address (ryanofsky)
  • 0 lines added or modified outside tests
  • 77 lines added or modified in tests
  • 2 lines removed in total
  • 6 files, 1 commits
Build and CI#22 of 56
P3 · test coverage
  • P3 because automated checks catch unintended external network queries during CI runs
  • Enforces CI hermeticity across all test stages without affecting core build outputs
  • Ranking pass: Moved down from 12: valuable CI hermeticity check but no observed incident, so it ranks below concrete release and depends fixes
Ranking notes
Test infrastructure#11 of 54
P3 · test coverage
  • P3 because it ensures isolation across all test suites by catching leaked packets and DNS requests
  • Adds valuable leak detection but is non-critical and deferrable
Ranking notes
Reviewability: ReadyStalePaused
Agreement: StrongPositivePositive w/ caveatsNeutralMildDisputedBlockedCrickets
Reviews: current code-review ACKs, then (+stale ACKs) and -NACKs; greener = more ACKs.
Size: lines added or modified outside tests, then in tests; greener = smaller.

Goal

Runs `tcpdump` during CI test execution to detect unexpected outbound TCP, UDP, or DNS network traffic. If any test stage (unit, functional, tidy, IWYU, fuzz) emits outbound packets on non-loopback interfaces or local DNS queries on port 53, the CI step dumps the captured packets and fails.

Problem: Tests running in CI can inadvertently generate outbound network traffic or DNS queries to the internet (for instance, via node initialization, DNS seeds, or RPC network calls), violating test hermeticity and potentially leaking test environment data or causing flakiness.

Evidence of importance:

  • Resolves issue #31339 regarding unwanted outbound network traffic during test runs.
  • Author notes that CI VMs can route DNS lookups externally: 'DNS traffic is detected because some VMs have configured a non-loopback DNS server, [1111:1111::1]:53. This is achieved by running tcpdump during tests and inspecting its output after the tests.'

Scope: The PR description mentions fixing `node_init_tests/init_test`, but the commit and diff modify only CI scripts and Docker options (`ci/test/*`), adding `tcpdump` and `net-tools` without touching test code.

Reviewability

Stale · Author silent 136d

No author activity for 136 days; a review may wait a long time for a response.

Waiting on (from the thread read):

  • author: address tcpdump --direction no-op flag in traffic monitor reading logic 2026-08-28

Agreement

Mild

objection that needs an answer is contested (fjahr)

Thread notes: Added ryanofsky's code review ACK (r:5268027561) and inline response (rc:4063256570) confirming fjahr's point that --direction is a no-op when reading capture files; the nonblocking claim remains open awaiting an author update.

Objections, suggestions, and questions:

ReviewerTypeHarm, request, or questionStatusMust beAuthor repliedQuote
laanwj (member)objection, approachFalse-positive CI failures if unrelated background daemons on the CI host VM generate network trafficanswered · laanwj did not follow up after author rationale · v1, not yet re-checkedansweredyes2024-11-25: 'i'm slightly worried this may generate false positive. As is, this detects traffic on the entire (virtual) machine while running the tests. Are there no other daemons running on the CI instance that c'
Status from: 2024-11-27: vasild: 'About the false positives - I think it is worth trying this in its current mode where any detected traffic is assumed to'
maflcko (member)objection, correctnesstcpdump buffers output when writing to a file, causing captured packets to be missed when the file is readfixed · fix pushed · Fixed by adding -U flag · v1, not yet re-checkedansweredyes2024-12-03: 'I don't think this works reliably? See: ... Use the -U flag to cause packets to be written as soon as they are received.'
Status from: 2024-12-04: vasild: 'Even better! Added `-U`.'
maflcko (member)objection, correctnessSilently ignoring return codes and omitting pipefail allows broken network detection to pass CI silentlyfixed · fix pushed · Pipefail enabled in CI scripts · v1, not yet re-checkedchangedyes2024-12-04: 'Ignoring the return code in all tasks can mean that even the tasks where the detection is working are going to silently break tomorrow. As mentioned previously in the thread you closed, I think this i'
Status from: 2024-12-05: vasild: 'Enabled `pipefail` as in the above diff. Thanks!'
fanquake (member)objection, approachCI scripts fail when run locally under Podman due to external DNS traffic, breaking supported local test workflowsfixed · fix pushed · Mitigated with -dnsseed=0 and test fix in #35193 · v1, not yet re-checkedchangedyes2025-10-16: 'Yea, I have seen the same, on one of my Fedora boxes, which would be a blocker, given it breaks running the CI locally.'
Status from: 2026-05-15: vasild: 'with the addition of `-dnsseed=0` this should be robust and work as intended on docker or podman, locally or in CI cloud'
maflcko (member)objection, correctnessNon-deterministic false-positive test failures during local CI runs under Podmanfixed · fix pushed · maflcko re-Concept ACKed after fixes · v1, not yet re-checkedchangedyes2026-02-04: 'The current failures are non-deterministic, which is a blocker to merging this. We have enough intermittent issues already, and I don't think it is a good use of time to add more, when there isn't eve'
Status from: 2026-05-15: vasild: 'with the addition of `-dnsseed=0` this should be robust and work as intended on docker or podman, locally or in CI cloud'
ryanofsky (member)objection, correctnessfilter variable is not reset for non-loopback interfaces and persists across invocations, which could ignore non-port-53 trafficfixed · fix pushed · Fixed variable resetting and scoping · v1, not yet re-checkedansweredyes2026-05-18: 'Looks like there is a bug here because `filter` variable is not reset if `ifname` does not begin with "lo". Also filter is not declared as a local variable so it will persist between calls.'
Status from: 2026-06-18: vasild: 'Done, thanks!'
fjahr (member)objection, correctnesstcpdump --direction is only enforced during packet capture and acts as a no-op when reading files, failing to filter directioncontested · ryanofsky agreed the flag is a no-op and suggested dropping or moving it; not yet addressed in code · v1, not yet re-checkedansweredby others2026-08-28: 'It seems that `--direction` is only enforced at capture time, so it should be moved to the tcpdump call in `traffic_monitor_begin()`. Or maybe just delete here if we don't want to filter there because'

Support:

  • maflcko (member) (Concept ACK, 2024-11-22): Nice. Conecpt ACK! [not substantive]
  • laanwj (member) (Concept ACK, 2024-11-25): Concept ACK [not substantive]
  • 1440000bytes (contributor) (Concept ACK, 2024-11-26): Concept ACK [not substantive]
  • jonatack (member) (Concept ACK, 2024-11-26): it turns out the owners of 1.2.3.4, 11.22.33.44 and 8.8.8.8, if they would bother, would know the IP address of every dev who runs the functional tests at home.
  • BrandonOdiwuor (contributor) (Concept ACK, 2024-11-27): Concept ACK [not substantive]
  • 0xB10C (member) (ACK, 2024-12-21) [tested]: Nice solution to use `127.0.0.1:1` as unreachable proxy for DNS requests too. I ran this on my CI runners with a 8.8.8.8 and 1.1.1.1 as DNS servers as before and the tests don't connect out now.
  • sipa (member) (Concept ACK, 2024-12-23): Concept ACK [not substantive]
  • fjahr (member) (Concept ACK, 2024-12-30): The changes in the tests look good but I need a little more time with the CI stuff. [not substantive]
  • Sjors (member) (Concept ACK, 2025-01-13) [tested]: I tested this with https://github.com/Sjors/bitcoin/pull/76 on my own CI setup, which uses Cirrus workers configured using (roughly) the instructions here: https://github.com/bitcoin/bitcoin/blob/mast [not substantive]
  • ryanofsky (member) (ACK, 2025-09-04) [read]: The new tcpdump mechanism to detect unexpected traffic is pretty simple and seems very nice.
  • ryanofsky (member) (ACK, 2025-09-09) [read]: Just rebased since last review and replaced node_init_tests fix CreateSock fix with natpmp=0 fix
  • ryanofsky (member) (ACK, 2025-09-29) [read]: just dropping INTERNET_TRAFFIC_EXPECTED variable since last review
  • fjahr (member) (Tested ACK, 2025-09-30) [tested]: Tested again by pushing the latest version of the code with the change of the first commit commented out. I could observe the expected failure in the CI this time
  • fjahr (member) (ACK, 2025-09-30) [read]: Just addressed @ryanofsky 's comments.
  • ryanofsky (member) (ACK, 2025-09-30) [read]: just documenting things better since last review and adding missing exit 1 to trigger CI failure (nice find!)
  • ryanofsky (member) (ACK, 2025-12-17) [read]: Since last review just rebased to avoid conflicts, added comments to commit message and test, added `local` to some bash variables
  • maflcko (member) (Concept ACK, 2026-05-05): I think there is value here, re-Concept ACK, to clarify. [not substantive]
  • ryanofsky (member) (ACK, 2026-09-21) [read]: Code review ACK 84bb496c6165154a64fcccd31e6ca5354531fd60. Since last review: rebased, fixed filter assignment and added local variable qualifiers. Would be good to address the `--direction` bug although the flag is harmless. Earlier concerns about running locally are resolved.

Participants:

  • maflcko (member, objection, 34 statements 2024-11-22 to 2026-05-05): Concept ACKed, but raised multiple objections regarding silent error ignoring and failures when running CI locally under Podman
  • laanwj (member, objection, 2 statements 2024-11-25 to 2024-11-25): Concept ACKed, but raised concerns that host background daemons could trigger false-positive CI failures
  • 0xB10C (member, support, 7 statements 2024-11-26 to 2025-01-08): ACKed an earlier commit after testing with non-loopback DNS servers in CI
  • 1440000bytes (contributor, support, 1 statements 2024-11-26 to 2024-11-26): Concept ACK
  • jonatack (member, support, 1 statements 2024-11-26 to 2024-11-26): Concept ACK, citing privacy risks of tests leaking developer IP addresses
  • BrandonOdiwuor (contributor, support, 1 statements 2024-11-27 to 2024-11-27): Concept ACK
  • sipa (member, support, 2 statements 2024-12-23 to 2025-09-09): Concept ACK
  • fjahr (member, objection, 8 statements 2024-12-30 to 2026-08-28): Previously tested and ACKed, but has an open objection regarding tcpdump --direction being a no-op when reading files
  • fanquake (member, objection, 7 statements 2025-01-06 to 2026-06-18): Opposed merging while local CI runs under Podman fail due to outbound network traffic
  • luke-jr (contributor, neutral, 1 statements 2025-01-08 to 2025-01-08): Suggested iptables UID matching
  • Sjors (member, support, 1 statements 2025-01-13 to 2025-01-13): Concept ACK and tested on Cirrus workers
  • ryanofsky (member, support, 13 statements 2025-09-04 to 2026-09-21): Code review ACKed latest revision, concurred --direction is a harmless no-op, previously reported a filter bug that was fixed
  • sedited (member, neutral, 1 statements 2026-08-27 to 2026-08-27): Pinged reviewers for follow-up reviews

Files

79 lines under test/bench/ci.

  • ci/test/03_test_script.sh +70/-0
  • ci/test/00_setup_env.sh +1/-1
  • ci/test/00_setup_env_mac_native.sh +2/-0
  • ci/test/00_setup_env_mac_native_fuzz.sh +2/-0
  • ci/test/00_setup_env_native_alpine_musl.sh +1/-1
  • ci/test/02_run_container.py +1/-0

Card

The short text the ranking pass compares across PRs.

Adds automated background network monitoring via tcpdump during CI test runs to fail builds if tests generate outbound TCP or UDP packets on non-loopback interfaces or make DNS queries. Without this check, regressions can slip into tests that inadvertently leak traffic or rely on live network access. The change resolves issue #31339 and protects unit, functional, fuzz, tidy, and IWYU CI steps.

Data

dossier JSON · extract JSON · model ledger, generated 2026-09-22T01:14, confidence high, input hash f2d955f9bb8011ec
thread read through 2026-09-21 (147 statements), head 84bb496c61
Discussion read under schema 1 rules: suggestions and questions may be missing, and an objection may combine several harms (schema history).
labels: Tests, Needs rebase

Categories

Build and CI

P3 · test coverage (ranking pass: Moved down from 12: valuable CI hermeticity check but no observed incident, so it ranks below concrete release and depends fixes)

This PR adds tcpdump monitoring to CI jobs to detect and fail on unexpected outbound TCP, UDP, or DNS traffic. Adding automated checks ensures test hermeticity and prevents CI runners from making external network queries.

Membership: Modifies ci/test/ scripts (00_setup_env*.sh, 02_run_container.py, 03_test_script.sh) to add tcpdump packet inspection to CI runs.

Factors: security/stability 1, bug 0, performance 0, user value 0, leverage 0

Test infrastructure

P3 · test coverage

It enforces isolation and hermeticity across all test suites by failing if tests emit outbound packets or non-loopback DNS queries. While worthwhile for detecting tests that leak requests to the public internet, it is non-critical and reasonably deferrable.

Membership: Carries the Tests label and introduces test execution monitoring across unit, functional, and fuzz test suites to detect network leaks.

Factors: security/stability 1, bug 0, performance 0, user value 0, leverage 0