Test infrastructure: ranking notes

Category notes

This category is dominated by P3 coverage additions of similar weight; I used the category's own ordering (signal fixes, then high-risk coverage, then framework capability, then speed, then cleanliness) to separate them rather than re-banding, and kept three P2s where the work is framework-defining or protects a supported-platform release path.

Review order and overlapping PRs

  • #35713 (replace Boost.Test runner), #35139 (new ASSERT/ASSERT_EQ macros) and #36091 (printers + BOOST_CHECK_EQUAL conversions) all rework unit-test assertions and diagnostics. #36091 is stated groundwork for #35713 and should be reviewed first; #35139's new macros partly overlap with what #35713 replaces, so the approach decision on #35713 should settle before #35139 is polished.
  • #36087 (AssertUnreachable in util/check.h plus linters) and #36074 (scripted-diff adding util/check.h includes across tests/bench/fuzz) are the same maintainer's assertion-header cleanup and will conflict; #36087 defines the primitive, so it comes first.
  • #36128, #35835 and #33973 all add or restructure script_tests vectors/JSON for the same interpreter flags; they will collide textually and are of equal consequence - review them as one batch rather than separately.
  • #34969 and #36068 both speed up brunoerg's wallet fuzz harnesses and touch overlapping FuzzedWallet code; #36068 (4-6x, target-wide init reuse) is the broader change and should go first, with #34969 rebased on it. #35342 also refactors shared RPC fuzz helpers that these targets touch.
  • #35442 contains only two of its own commits and sits on unmerged #35405/#35433; it cannot be reviewed independently of that chain.
  • #35675 is mostly a mining refactor (BlockTemplateManager, NodeContext changes) whose downstream dependents #33922/#35581 are mining PRs; its test-infrastructure content is incidental and it arguably belongs in the mining category, where its unblocking role would rank higher.
  • #36184 fixes a test-framework default that no in-tree caller exercises and maflcko suggested closing it or using None instead; it overlaps in intent with #36185's extendedkey helper work - a decision on the helper API should precede either.

Band and position changes

  • #36129: Moved up: order-dependent, non-reproducible fuzz state is a signal/trust problem, which this category ranks ahead of new coverage.
  • #36254: Moved up: a benchmark that silently measures the wrong code path is a misleading test, ranked ahead of added coverage here.
  • #36091: Moved up: supplies the printer/diagnostic infrastructure the #35713 runner overhaul builds on, so it inherits some of that work's leverage.
  • #31349: Slightly lower than its own score placed it: hermeticity/privacy check is valuable but no in-tree test is known to be leaking today.
  • #36068: Placed after the signal fixes; throughput gains rank below trust-in-CI fixes in this category.
  • #33954: Lowered a few places: real framework capability, but author states it is not ready and test intermittency is unresolved, so benefit is less certain than the speed/signal items above.
  • #32993: Lowered: new wallet-rescan fuzz coverage, but paused in draft pending a harness rewrite, making it less certain than the framework/speed items above.
  • #36135: Grouped with the other remote-input fuzz coverage; reviewers mutation-verified it, so it leads that group.
  • #33973: Grouped with the other script_tests.json vector additions of equal consequence.
  • #35762: Lowered: workflow ergonomics only, and reviewers dispute whether the added runner logic is warranted.
  • #34360: Lowered substantially: a single wallet benchmark serving one candidate optimization (#27865) is narrower than the other P3 work here.
  • #35177: Lowered: real fix (#31838) but only affects the --gen-test-data maintenance path on no-wallet builds, not normal test runs.
  • #35686: Top of P4: the linter prevents a concrete recurring maintainer cost on subtree backports, unlike the purely cosmetic items below.
  • #35342: P3 alone, P4 after comparison. Lowered to P4: reviewers state fuzzing wallet RPCs without chain state or UTXOs has very low efficacy, so the coverage claim is weakly supported compared with the P3 fuzz harnesses here.
  • #35442: Lowered: only two of thirteen commits belong to it, it depends on unmerged base PRs, and reviewers question whether it should exist separately.
  • #32409: Placed last: the variable is set nowhere in-tree, MSVC debug builds are not used in CI, and the change is NACKed as unused code.

About

This pass by claude-opus-5 on 2026-09-17 saw every PR in the category at once and checked the bands given to each PR alone against each other, ordered the PRs, and noted chains and overlaps. Back to the category.