{ "number": 36350, "repo": "bitcoin/bitcoin", "url": "https://github.com/bitcoin/bitcoin/pull/36350", "title": "net_processing: count presync work in chain sync eviction", "author": "0xShadowX", "author_association": "FIRST_TIME_CONTRIBUTOR", "created_at": "2026-09-26T22:33:08Z", "updated_at": "2026-10-22T07:04:39Z", "description_hash": "5a92836cb2fbc1cb", "age_days": 36, "draft": false, "labels": [ "P2P", "CI failed" ], "milestone": null, "base": "master", "head_sha": "d59713ea518655031e2160026d85316ae19900fe", "head_ref": "fix-presync-chain-sync-eviction", "head_repo": "0xShadowX/bitcoin", "head_history": [], "additions": 192, "deletions": 2, "changed_files": 3, "commit_count": 4, "size_bucket": "M", "mergeable_state": "unknown", "bot": { "drahtbot": { "present": true, "reviews": { "ack": [ { "login": "cyb3ralbert", "url": "https://github.com/bitcoin/bitcoin/pull/36350#issuecomment-6334875534" } ] }, "conflicts": [] } }, "acks_parsed": { "cyb3ralbert": { "kind": "ack", "hash": "d59713ea518655031e2160026d85316ae19900fe", "t": "2026-10-22T07:04:35Z", "stale": false } }, "acks_tally": { "ack": 1, "stale_ack": 0, "concept_ack": 0, "approach_ack": 0, "nack": 0, "concept_nack": 0, "approach_nack": 0 }, "reviews": { "approved": 0, "changes_requested": 0, "distinct_reviewers": [ "cyb3ralbert" ] }, "signals": { "needs_rebase": false, "ci_failed": true, "mergeable_state": "unknown", "last_author_activity": "2026-10-21T08:49:04Z", "last_reviewer_activity": "2026-10-22T07:04:35Z", "last_reviewer": "cyb3ralbert", "author_silent_days": 11, "waiting_on_author_days": 10, "days_since_update": 10 }, "refs": { "mentioned": [ 11490, 25717 ], "depends_on": [], "fixes": [], "linked_issues": [], "references": [ { "number": 11490, "type": "pull", "state": "closed", "merged": true, "merged_at": "2017-10-26", "title": "Disconnect from outbound peers with bad headers chains" }, { "number": 25717, "type": "pull", "state": "closed", "merged": true, "merged_at": "2022-08-30", "title": "p2p: Implement anti-DoS headers sync" } ], "conflicts": [] }, "stack": { "shares_commits_with": [], "based_on": [], "base_for": [] }, "review_paths": [], "body": "While our best header is below `-minimumchainwork`, a peer's headers first go through the PRESYNC phase of `HeadersSyncState`, and none of them are stored. So `pindexBestKnownBlock` stays null for the peer we're syncing from, and `ConsiderEviction` (net_processing.cpp:5504) treats it like an outbound peer that never announced a block:\n\n1. it starts `CHAIN_SYNC_TIMEOUT` (20 min)\n2. it sends a getheaders to \"verify chain work\"\n3. after `HEADERS_RESPONSE_TIME` it disconnects with `Outbound peer has old chain, best known block = `\n\nThis happens even though the peer is still sending headers. The presync progress is lost and the next peer starts from scratch.\n\nOn mainnet presync is roughly 480 headers round trips. If that takes more than ~22 minutes (about 2.5 s per round trip, e.g. over slow Tor or I2P), the sync peer gets evicted before presync can finish, and the same thing can happen with the next peer. The chain sync logic is from #11490, presync came later in #25717, and the two were never connected.\n\nThe timeout is not reset on presync work alone. When it is set, the work the peer has shown during presync is recorded, and when it expires the timeout is renewed for another `CHAIN_SYNC_TIMEOUT` only if that work has increased since. A peer that stops sending headers is therefore still asked once with getheaders and then disconnected, within about two periods (~42 minutes), as before. `m_headers_sync_mutex` is never held while taking `cs_main`, and `p2p_headers_sync_with_minchainwork.py` and `p2p_sendheaders.py` also pass with `DEBUG_LOCKORDER`.\n\nThe new `test_presync_peer_not_evicted` in `p2p_headers_sync_with_minchainwork.py` restarts a node with `-minimumchainwork=0x20000` (65536 headers). An outbound peer then sends it one full headers message per mocked minute for 25 minutes, so presync is still running at the end. On master the peer is disconnected after 24 minutes. With the fix it stays connected and `presynced_headers` keeps going up. `p2p_outbound_eviction.py`, `p2p_eviction.py` and `feature_minchainwork.py` still pass.\n\n`test_presync_peer_stalled_is_evicted` covers the other side: a single outbound peer sends one headers message and then only answers pings, and it is disconnected after about 42 minutes of mocktime. That test fails without the progress check.", "commits": [ { "sha": "34ef37a550b1deb6d46a92c8ac4b24ab62baddf3", "date": "2026-09-26T22:31:27Z", "message": "net_processing: count presync work in chain sync eviction\n\nWhile the node's best header has less work than -minimumchainwork, a\npeer's headers go through the PRESYNC phase first, and none of them are\nstored. pindexBestKnownBlock stays null for that peer, so\nConsiderEviction treats the sync peer like an outbound peer that never\nannounced anything: it starts the CHAIN_SYNC_TIMEOUT, then disconnects\nit with \"Outbound peer has old chain\" after about 22 minutes, even\nthough the peer is still delivering headers. Presync progress is lost,\nand the next sync peer starts over.\n\nOn mainnet presync takes about 480 round trips, so this hits when\nheaders download is slow (e.g. slow Tor or I2P links).\n\nTreat a peer as caught up when the work it has shown during presync is\nat least our tip's work." }, { "sha": "bf56f4fb8328b594a7346c92487dba825a32fb36", "date": "2026-10-14T08:02:18Z", "message": "net_processing: only reset the chain sync timeout on presync progress\n\nResetting the timeout whenever the presync work was at least our tip's\nwork let a peer that stopped serving headers stay connected indefinitely.\nRecord the presync work when the timeout is set and renew the timeout\nonly if the work increased by the time it expires. A stalled peer is\nasked for headers once and disconnected as before." }, { "sha": "69e0e718862d266e8e9ebc2ad77b4b303f368f55", "date": "2026-10-15T07:36:19Z", "message": "net_processing: count headers redownload progress and bound renewals\n\nOnce presync reaches the minimum chain work, GetPresyncWork() stops\ngrowing while the redownload buffer holds headers back, so a slow peer\ncould not renew the chain sync timeout during that phase. Track progress\nwith the number of headers accepted in either phase instead.\n\nRenewing on every bit of progress also let a peer that sends one headers\nmessage per period hold the outbound slot indefinitely. Limit the\nrenewals to MAX_HEADERS_SYNC_TIMEOUT_RENEWALS.\n\nMake the stalled test sync with a ping after every mocktime step, and\nadd tests for redownload progress and for the renewal bound." }, { "sha": "d59713ea518655031e2160026d85316ae19900fe", "date": "2026-10-21T08:48:58Z", "message": "test: pin the number of headers sync timeout renewals\n\ntest_headers_sync_renewals_bounded only checked that the peer is eventually disconnected, so it also passed with MAX_HEADERS_SYNC_TIMEOUT_RENEWALS set to 5 or 7. Check that the peer is still connected after 7 periods and gone after 8." } ], "timeline": [ { "t": "2026-10-14T04:57:34Z", "kind": "comment", "who": "cyb3ralbert", "assoc": "CONTRIBUTOR", "text": "Tested 34ef37a (also merged with master 8406af1c).\n\nThe new test passes on this PR but fails on master. With a single outbound peer, a peer that sends one headers message and then stops responding to `getheaders` remains connected for 24 hours under mocktime, whereas master disconnects it after 24 minutes. The peer still responds to pings.\n\nWith a second outbound peer, the headers download timeout fires after 29 minutes, for the slow peer as well.\n\nCould we reset the timeout only when presync work has increased since the previous timeout checkpoint? This might preserve the intended behavior for slow peers without allowing a stalled peer to hold the only outbound slot.\n\nA couple of questions:\n- Is the Tor/I2P latency scenario expected to apply when this is the only outbound peer?\n- Was the eviction observed on a real node, or inferred from expected round-trip times?", "id": "c:6161235415", "url": "https://github.com/bitcoin/bitcoin/pull/36350#issuecomment-6161235415", "hash": "7d8bc7af6f6d2941", "edited": null }, { "t": "2026-10-14T08:02:40Z", "kind": "comment", "who": "0xShadowX", "assoc": "NONE", "text": "Thanks for testing, and you're right: resetting whenever presync work was at least the tip's work let a stalled peer hold the slot indefinitely.\n\nPushed bf56f4fb83. The presync work is now recorded when the timeout is set, and when it expires the timeout is only renewed if that work has increased since. Otherwise it goes through the usual getheaders and disconnect. A new test, `test_presync_peer_stalled_is_evicted`, sends one headers message and then only answers pings; the peer is disconnected after ~42 minutes of mocktime (one renewal because of the first headers, then the normal timeout), and the test fails on the previous revision.\n\nOn the questions:\n- Single outbound peer: nothing in the logic depends on how many outbound peers there are, so a slow Tor/I2P link can hit it with one peer as well, and then it has no other peer to fall back on. I haven't measured that case.\n- Real node: no, the eviction was not observed on a real node. It is inferred from the ~480 presync round trips on mainnet against the ~22 minute window, and reproduced with mocktime in the functional test.", "id": "c:6163775056", "url": "https://github.com/bitcoin/bitcoin/pull/36350#issuecomment-6163775056", "hash": "f7d9b9857625d6b1", "edited": null }, { "t": "2026-10-15T05:32:00Z", "kind": "comment", "who": "cyb3ralbert", "assoc": "CONTRIBUTOR", "text": "Once presync reaches the minimum chain work and `HeadersSyncState` switches to REDOWNLOAD, `GetPresyncWork()` stops growing. As a result, the new branch cannot renew the timeout until the redownload buffer releases headers and `pindexBestKnownBlock` advances. On mainnet, this means 15,400 headers (8 full messages); on regtest, 7,017 headers (4 messages).\n\nA peer that needs timeout renewals during presync therefore has only 22\u201342 minutes after the transition to deliver those headers before being disconnected. `HeadersSyncState` already tracks `m_redownload_chain_work` and `m_redownload_buffer_last_height`; should progress tracked by these fields also count towards timeout renewal?\n\nTested bf56f4fb8, also merged with master f13ae519d.\n\nThe regression from my previous comment is fixed: `test_presync_peer_stalled_is_evicted` fails on 34ef37a55 and passes here. With one full headers message followed only by pongs, the peer is disconnected 42 min 15 s after connecting (master: 22 min 10 s).\n\nThe runs behind the REDOWNLOAD question, on regtest with one outbound peer, mocktime, and `-minimumchainwork=0x20000`:\n\n- One full headers message every 10 minutes reaches the transition at 66,000 headers, then the peer is disconnected 40 minutes later with `Outbound peer has old chain, best known block = `.\n- Every 19 minutes: disconnected 25 minutes after the transition.\n- Every 1 or 5 minutes: sync completes.\n- On master, the peer sending one message per minute is disconnected at minute 24.\n\nA separate question is repeated renewals. With `-minimumchainwork=0x40000`, one full message every 19 minutes kept the peer connected for 8 hours, while every 25 minutes resulted in disconnection at minute 150. There does not appear to be a bound on the number of renewals, so a peer making very slow but regular progress can hold the outbound slot for an extended period.\n\nIs this the intended trade-off? Should renewals be bounded, or should the timeout account for progress without allowing indefinitely repeated extensions?\n\nA note on the stalled test: its loop advances mocktime by one minute per 0.2 seconds without synchronizing pings, and the timeout comparisons are strict. The expected sequence is renewal at minute 21, `getheaders` at minute 42, and disconnection at minute 45 (the last iteration). A late `SendMessages` pass at minute 21 or 42 can push the timeout beyond minute 45, leaving mocktime stuck there until `wait_for_disconnect` times out. Calling `sync_with_ping()` on each step, or allowing a few extra minutes, would make the test less timing-sensitive.\n\n`test_presync_peer_not_evicted` fails on master and passes here. `p2p_headers_sync_with_minchainwork.py`, `p2p_eviction.py`, `p2p_outbound_eviction.py`, `p2p_initial_headers_sync.py`, and `feature_minchainwork.py` pass on the head, the head merged with master, and a clang Debug build with `DEBUG_LOCKORDER`.\n\nNot tested: mainnet round-trip times, Tor/I2P, or multiple outbound peers.", "id": "c:6185199965", "url": "https://github.com/bitcoin/bitcoin/pull/36350#issuecomment-6185199965", "hash": "2d8938b54109fb89", "edited": null }, { "t": "2026-10-15T07:36:34Z", "kind": "comment", "who": "0xShadowX", "assoc": "NONE", "text": "Thanks for the detailed testing.\n\nREDOWNLOAD: agreed. The renewal now uses a progress counter that covers both phases (`HeadersSyncState::GetSyncProgress()`, presync height + redownload height), so progress in the redownload phase also counts. New test `test_redownload_peer_not_evicted` sends one message every 15 minutes after the transition; it fails on bf56f4fb8 and passes on 69e0e7188.\n\nRenewals: now bounded by `MAX_HEADERS_SYNC_TIMEOUT_RENEWALS = 6`, so a peer can hold the slot for at most ~2h22m through this path. `test_headers_sync_renewals_bounded` (one message every 19 minutes) is disconnected after 7 periods. The trade-off is that a peer has to finish both phases within that window; on mainnet that is ~950 messages, i.e. ~7 messages/min (~150 kbps). I picked 6 as a value that leaves plenty of room for slow links while bounding the slot; happy to change it if a different bound (or a rate-based rule) is preferred.\n\nStalled test: `advance_minutes` now calls `sync_with_ping()` after every mocktime step and the loop runs to 50 minutes.", "id": "c:6187005712", "url": "https://github.com/bitcoin/bitcoin/pull/36350#issuecomment-6187005712", "hash": "29462de94843b22a", "edited": null }, { "t": "2026-10-21T07:44:40Z", "kind": "comment", "who": "cyb3ralbert", "assoc": "CONTRIBUTOR", "text": "One gap in the new bound test: `test_headers_sync_renewals_bounded` doesn't pin the number of renewals. With `MAX_HEADERS_SYNC_TIMEOUT_RENEWALS` set to 5 or to 7 the whole file still passes; the test peer is then disconnected at minute 129 or 171 instead of 150. Two ways to pin it, both tried on 69e0e7188 (Linux x86_64, GCC 14.2, with 5, 6 and 7): asserting inside the existing loop that the peer is still connected after 7 steps of 19 minutes (minute 133) and gone after 8 (minute 152) fails on 5 and on 7 and passes on 6; and, since the renewal branch in `ConsiderEviction` logs nothing, adding one `LogDebug` there and asserting exactly 6 matches in `debug.log` fails on 5 (saw 5) and on 7 (saw 7) and passes on 6. The second one checks the renewal count itself rather than the timing. I can send either as a diff if useful.\n\nTested 69e0e7188 (Linux x86_64, GCC 14.2 and 12.2, Release build, regtest, mocktime, one outbound peer):\n\n- `test_redownload_peer_not_evicted` fails on bf56f4fb8 (`assert peer.is_connected`) and passes on 69e0e7188; `p2p_headers_sync_with_minchainwork.py` passes as a whole (3 runs).\n- The bound matches your ~2h22m. A peer that keeps serving a full 2000-header message every mocktime minute, with `-minimumchainwork` out of reach so it stays in presync, is disconnected at minute 150 with 300,000 headers; by my count the extra minutes come from the one-minute mocktime steps, since the comparisons are strict. On bf56f4fb8 the same peer is still connected at minute 160.", "id": "c:6310806124", "url": "https://github.com/bitcoin/bitcoin/pull/36350#issuecomment-6310806124", "hash": "2d971de4b74c53ee", "edited": null }, { "t": "2026-10-21T08:49:04Z", "kind": "comment", "who": "0xShadowX", "assoc": "NONE", "text": "Thanks, good catch. Pushed d59713ea51: the loop in `test_headers_sync_renewals_bounded` now asserts the peer is still connected after 7 periods of 19 minutes (minute 133) and gone after the 8th (minute 152). I ran the file with `MAX_HEADERS_SYNC_TIMEOUT_RENEWALS` at 5, 6 and 7: it passes on 6 and fails on 5 and 7. I went with the timing check so no production code is touched for a log line.", "id": "c:6311936322", "url": "https://github.com/bitcoin/bitcoin/pull/36350#issuecomment-6311936322", "hash": "a0ad38791e6d1902", "edited": null }, { "t": "2026-10-22T07:04:35Z", "kind": "comment", "who": "cyb3ralbert", "assoc": "CONTRIBUTOR", "text": "ACK d59713ea518655031e2160026d85316ae19900fe\n\nThe new commit only touches the functional test (no src/ change). `p2p_headers_sync_with_minchainwork.py` passes. With `MAX_HEADERS_SYNC_TIMEOUT_RENEWALS` set to 5 `test_headers_sync_renewals_bounded` fails at `assert peer.is_connected`, and with 7 it fails at `peer.wait_for_disconnect()`; with 6 it passes, so the bound is now pinned from both sides.\n\nTested on Debian 13 x86_64, gcc 14.2.0, Release build.", "id": "c:6334875534", "url": "https://github.com/bitcoin/bitcoin/pull/36350#issuecomment-6334875534", "hash": "5ad9ca47db737e52", "edited": null } ], "labels_log": [ { "t": "2026-09-28T06:49:45Z", "action": "labeled", "label": "P2P", "who": "maflcko" }, { "t": "2026-09-28T12:06:18Z", "action": "labeled", "label": "CI failed", "who": "DrahtBot" }, { "t": "2026-09-29T19:17:43Z", "action": "unlabeled", "label": "CI failed", "who": "DrahtBot" }, { "t": "2026-10-20T07:18:57Z", "action": "labeled", "label": "CI failed", "who": "DrahtBot" } ], "state_log": [], "text_chars": 12875, "text_tokens_estimate": 3218, "changed_paths": [ "src/headerssync.h", "src/net_processing.cpp", "test/functional/p2p_headers_sync_with_minchainwork.py" ], "files": [ { "path": "src/headerssync.h", "add": 4, "del": 0 }, { "path": "src/net_processing.cpp", "add": 27, "del": 1 }, { "path": "test/functional/p2p_headers_sync_with_minchainwork.py", "add": 161, "del": 1 } ], "test_lines": 162, "git": { "head": "d59713ea518655031e2160026d85316ae19900fe", "head_matches_backup": true, "base": "ed7dd7cf4e1561a97edf72eba29a67b14e28c717", "patch_id": "fc0bf2646b576486", "commits": [ { "sha": "34ef37a550", "subject": "net_processing: count presync work in chain sync eviction", "files": 2, "add": 73, "del": 2 }, { "sha": "bf56f4fb83", "subject": "net_processing: only reset the chain sync timeout on presync progress", "files": 2, "add": 60, "del": 15 }, { "sha": "69e0e71886", "subject": "net_processing: count headers redownload progress and bound renewals", "files": 3, "add": 129, "del": 56 }, { "sha": "d59713ea51", "subject": "test: pin the number of headers sync timeout renewals", "files": 1, "add": 6, "del": 5 } ], "patch_truncated": false }, "patch_id": "fc0bf2646b576486", "input_hash": "88704403c6774f45", "extracted_at": "2026-11-02T01:09:49+00:00" }