{ "number": 35837, "repo": "bitcoin/bitcoin", "url": "https://github.com/bitcoin/bitcoin/pull/35837", "title": "rpc: fail scanblocks when block filter range is unavailable", "author": "MicSm", "author_association": "FIRST_TIME_CONTRIBUTOR", "created_at": "2026-07-29T18:17:55Z", "updated_at": "2026-09-17T07:31:19Z", "age_days": 49, "draft": false, "labels": [ "RPC/REST/ZMQ" ], "milestone": null, "base": "master", "head_sha": "55abc8b820264b5143e1be31e4ef6b8e73f3b4ff", "head_ref": "rpc/scanblocks-require-synced-index", "head_repo": "MicSm/bitcoin", "head_history": [ { "t": "2026-07-31T11:17:18Z", "sha": "120b8e4299c3e1bea2cb6d44a69fbf32f42d9fe9" }, { "t": "2026-08-05T15:55:45Z", "sha": "55abc8b820264b5143e1be31e4ef6b8e73f3b4ff" } ], "additions": 81, "deletions": 13, "changed_files": 2, "commit_count": 1, "size_bucket": "S", "mergeable_state": "clean", "bot": { "drahtbot": { "present": true, "reviews": { "concept_ack": [ { "login": "maflcko", "url": "https://github.com/bitcoin/bitcoin/pull/35837#issuecomment-5127611820" } ] }, "conflicts": [ { "number": 36257, "title": "qa: assert_equals -> assert_true/assert_false", "author": "hodlinator" }, { "number": 35229, "title": "refactor: Use CBlockIndex parameters as reference", "author": "optout21" } ] } }, "acks_parsed": {}, "acks_tally": { "ack": 0, "stale_ack": 0, "concept_ack": 0, "approach_ack": 0, "nack": 0, "concept_nack": 0, "approach_nack": 0 }, "reviews": { "approved": 0, "changes_requested": 0, "distinct_reviewers": [ "fjahr", "maflcko" ] }, "signals": { "needs_rebase": false, "ci_failed": false, "mergeable_state": "clean", "last_author_activity": "2026-08-05T15:56:19Z", "last_reviewer_activity": "2026-07-30T20:47:45Z", "last_reviewer": "fjahr", "author_silent_days": 43, "waiting_on_author_days": 0, "days_since_update": 0 }, "refs": { "mentioned": [], "depends_on": [], "fixes": [], "linked_issues": [], "references": [], "conflicts": [ 36257, 35229 ] }, "stack": { "shares_commits_with": [], "based_on": [], "base_for": [] }, "review_paths": [ "src/rpc/blockchain.cpp" ], "body": "### The issue\n\n`scanblocks` reads block filters over a range via `LookupFilterRange`. When the\nblock filter index is behind the active chain (e.g. right after startup, while\nit is still syncing in the background), the lookup for the not yet indexed range\nfails. The old code ignored that failure, advanced `start_index` to the end of\nthe chunk, and kept going - so the scan skipped every unindexed block and still\nreturned `\"completed\": true`.\n\nThe result is a silent gap: a caller gets a `relevant_blocks` list that looks\nfine but is missing any match in the unindexed range, with no way to\ndistinguish \"no matches\" from \"range was never scanned\".\n\nNote that `\"completed\"` only means the scan was not aborted - it does not mean\nevery filter in the requested range was actually read.\n\n### Steps to reproduce\n\n1. Start a node with `-blockfilterindex=1` and let it mine/receive some blocks.\n2. Stop the node, delete `//indexes/blockfilter`, restart with\n `-blockfilterindex=1`.\n3. Immediately (before the index finishes rebuilding) call\n `scanblocks start '[\"addr()\"]'`.\n\nBefore this change the call returns `\"completed\": true` with an empty/partial\n`relevant_blocks`, silently skipping the range the index had not rebuilt yet.\n\n### How it is fixed and why\n\nMirror `getblockfilter`'s error classification:\n\n- Call `BlockUntilSyncedToCurrentChain()` only to obtain an `index_ready` flag.\n Do **not** fail unconditionally when the index is still behind the tip - the\n requested range may already be available.\n- If `LookupFilterRange` fails and the index is not ready, throw the same\n \"still in the process of being indexed\" `RPC_MISC_ERROR` that\n `getblockfilter` uses.\n- If `LookupFilterRange` fails after the index is ready, throw\n `RPC_INTERNAL_ERROR` (unexpected / corruption), instead of skipping the chunk.\n\nThis closes the silent gap without rejecting scans whose range is already\ncovered while the index catches up to the tip.\n\nThe diff also reindents the match loop: inverting `if (LookupFilterRange(...))`\ninto an early `throw` removes one nesting level from the existing body. That\nreindentation is a consequence of the bug fix, not a standalone style change.\n\n### Tests\n\n- `test/functional/rpc_scanblocks.py` - new `test_scanblocks_unindexed_range`\n covers the modified code: it pads the chain, wipes the filter index, restarts,\n and while the index is behind the tip asserts that a scan to the tip returns\n the indexing error, that a scan over an already-written prefix (genesis)\n still succeeds, and that after sync the match is found.\n- The existing `run_test` cases (which `wait_until(... synced ...)` before\n scanning) continue to exercise the normal synced path.\n\n### AI / tooling note\n\nI found this regression while reviewing the RPC with an AI-assisted tool.\nI reproduced the unsynced-index behavior myself, chose to mirror\n`getblockfilter`'s existing guard, and verified the fix with the\nfunctional test above. This PR text and any review replies are my own.", "commits": [ { "sha": "55abc8b820264b5143e1be31e4ef6b8e73f3b4ff", "date": "2026-08-05T15:30:23Z", "message": "rpc: fail scanblocks when block filter range is unavailable\n\nscanblocks reads block filters over a range with LookupFilterRange. A\nfailed read was ignored, so the scan skipped the affected blocks and\nstill returned \"completed\": true, hiding the gap from the caller.\n\nReport the failure instead. As getblockfilter does, it is returned as\n\"still being indexed\" while the index is behind the tip and as an\ninternal error once synced, so a scan whose range is already available\nkeeps working.\n\nAdd functional test coverage for the unindexed-tip error and for a\nsuccessful scan over an already-indexed range." } ], "timeline": [ { "t": "2026-07-30T06:52:40Z", "kind": "comment", "who": "maflcko", "assoc": "MEMBER", "text": "[quoted text omitted]\n\nThx for following the projects AI policy, and Concept ACK. However, I think the link to the tool can be removed and this can just say \"I found this regression while reviewing the RPC with an AI-assisted tool\". The tool is \"a layer for agents\", but I am sure any vanilla agent will find the same issue with the same prompt. E.g:\n\nLLM output\n\n```\ncdx -c model=gpt-5.4-mini exec 'Review the scanblocks RPC for correctness' 2>/dev/null\nFindings:\n\n1. [`src/rpc/blockchain.cpp`](src\\/rpc\\/blockchain.cpp#L2683) silently ignores `LookupFilterRange()` failures. If the block filter DB read fails or the range is otherwise unreadable, the RPC just skips that chunk, keeps going, and can still return `completed: true` with missing `relevant_blocks`. That is a correctness bug, not just an error-path detail. It should surface the failure to the caller or at least mark the scan incomplete.\n\n2. [`src/rpc/blockchain.cpp`](src\\/rpc\\/blockchain.cpp#L2664) has an abort race during startup. `g_scanfilter_should_abort_scan` is cleared only after the scan has already reserved the global slot and finished descriptor/index setup. If another thread issues `scanblocks(\"abort\")` in that window, it can set the flag and then have it overwritten by the startup path, causing the scan to ignore a valid abort request. The reset needs to happen before any long setup, or the state transition needs to be synchronized.\n\nNo other correctness issue stood out in the range logic itself.\n```" }, { "t": "2026-07-30T20:47:21Z", "kind": "review_comment", "who": "fjahr", "assoc": "MEMBER", "path": "src/rpc/blockchain.cpp", "commit": "9467446a9f1ef355e5525a469728340713d7cce4", "in_reply_to": null, "text": "This is incorrect, `getblockfilter` does not do this: https://github.com/bitcoin/bitcoin/blob/67efced1fc83a0b7215cc1513e7c4754fee0f12f/src/rpc/blockchain.cpp#L3008" }, { "t": "2026-07-30T20:47:45Z", "kind": "review", "who": "fjahr", "assoc": "MEMBER", "state": "COMMENTED", "commit": "9467446a9f1ef355e5525a469728340713d7cce4", "text": "[quoted text omitted]\n\nWhat this is doing here is different from what `getblockfilter` does. There we don't fail unconditionally if the index isn't synced. We only fail if we the range isn't available. This makes sense because we may have the range available even if the index isn't synced to the tip yet. I don't see why we shouldn't be able to do it the same way here." }, { "t": "2026-07-31T11:17:18Z", "kind": "force_push", "who": "MicSm", "commit": "120b8e4299c3e1bea2cb6d44a69fbf32f42d9fe9" }, { "t": "2026-08-04T16:29:33Z", "kind": "comment", "who": "MicSm", "assoc": "NONE", "text": "[quoted text omitted]\n\nYou're right - thanks. `getblockfilter` does not fail merely because index is behind the tip, so it fails when the requested filter isn't available (and then classifies it still-indexing vs unexpected/corruption).\n\nSo, I updated to match this: `BlockUntilSyncedToCurrentChain()` is only used as an `index_ready` flag and a failed `LookupFilterRange` is classified the same way. A scan whose requested range is already available while the tip\nis ahead, now succeeds." }, { "t": "2026-08-05T15:55:45Z", "kind": "force_push", "who": "MicSm", "commit": "55abc8b820264b5143e1be31e4ef6b8e73f3b4ff" }, { "t": "2026-08-05T15:56:19Z", "kind": "comment", "who": "MicSm", "assoc": "NONE", "text": "[quoted text omitted]\n\ndone" } ], "labels_log": [ { "t": "2026-07-29T18:17:59Z", "action": "labeled", "label": "RPC/REST/ZMQ", "who": "DrahtBot" }, { "t": "2026-07-30T20:51:06Z", "action": "labeled", "label": "CI failed", "who": "DrahtBot" }, { "t": "2026-08-05T11:52:19Z", "action": "labeled", "label": "Needs rebase", "who": "DrahtBot" }, { "t": "2026-08-05T16:15:02Z", "action": "unlabeled", "label": "Needs rebase", "who": "DrahtBot" }, { "t": "2026-08-05T23:47:52Z", "action": "unlabeled", "label": "CI failed", "who": "DrahtBot" } ], "state_log": [ { "t": "2026-07-31T07:45:50Z", "kind": "renamed", "who": "MicSm", "from": "rpc: fail scanblocks when block filter index is not caught up", "to": "rpc: fail scanblocks when block filter range is unavailable" } ], "text_chars": 6219, "text_tokens_estimate": 1554, "changed_paths": [ "src/rpc/blockchain.cpp", "test/functional/rpc_scanblocks.py" ], "files": [ { "path": "src/rpc/blockchain.cpp", "add": 22, "del": 13 }, { "path": "test/functional/rpc_scanblocks.py", "add": 59, "del": 0 } ], "test_lines": 59, "git": { "head": "55abc8b820264b5143e1be31e4ef6b8e73f3b4ff", "head_matches_backup": true, "base": "c4fbd3c72114e3466329ec6f4b0f0a4b7ebce3b1", "commits": [ { "sha": "55abc8b820", "subject": "rpc: fail scanblocks when block filter range is unavailable", "files": 2, "add": 81, "del": 13 } ], "patch_truncated": false }, "input_hash": "5dd2e0fbf518f110", "extracted_at": "2026-09-17T16:15:31+00:00" }