{
 "number": 36049,
 "repo": "bitcoin/bitcoin",
 "url": "https://github.com/bitcoin/bitcoin/pull/36049",
 "title": "streams: avoid termination on buffered write failure",
 "author": "l0rinc",
 "author_association": "MEMBER",
 "created_at": "2026-08-20T22:06:29Z",
 "updated_at": "2026-09-05T20:38:19Z",
 "age_days": 27,
 "draft": false,
 "labels": [],
 "milestone": null,
 "base": "master",
 "head_sha": "d5f424103bfb51a5aae3f91b023f1c23742e16d0",
 "head_ref": "l0rinc/streams-buffered-write-failure",
 "head_repo": "l0rinc/bitcoin",
 "head_history": [],
 "additions": 20,
 "deletions": 3,
 "changed_files": 3,
 "commit_count": 3,
 "size_bucket": "S",
 "mergeable_state": "clean",
 "bot": {
  "drahtbot": {
   "present": true,
   "reviews": {
    "concept_ack": [
     {
      "login": "w0xlt",
      "url": "https://github.com/bitcoin/bitcoin/pull/36049#issuecomment-5554558549"
     }
    ]
   },
   "conflicts": []
  }
 },
 "acks_parsed": {
  "w0xlt": {
   "kind": "concept_ack",
   "hash": null,
   "t": "2026-09-05T20:25:47Z",
   "stale": false
  }
 },
 "acks_tally": {
  "ack": 0,
  "stale_ack": 0,
  "concept_ack": 1,
  "approach_ack": 0,
  "nack": 0,
  "concept_nack": 0,
  "approach_nack": 0
 },
 "reviews": {
  "approved": 0,
  "changes_requested": 0,
  "distinct_reviewers": [
   "andrewtoth",
   "maflcko",
   "w0xlt"
  ]
 },
 "signals": {
  "needs_rebase": false,
  "ci_failed": false,
  "mergeable_state": "clean",
  "last_author_activity": "2026-08-20T22:35:20Z",
  "last_reviewer_activity": "2026-09-05T20:25:47Z",
  "last_reviewer": "w0xlt",
  "author_silent_days": 27,
  "waiting_on_author_days": 11,
  "days_since_update": 11
 },
 "refs": {
  "mentioned": [
   31551
  ],
  "depends_on": [],
  "fixes": [],
  "linked_issues": [],
  "references": [
   {
    "number": 31551,
    "type": "pull",
    "state": "closed",
    "merged": true,
    "merged_at": "2025-04-16",
    "title": "[IBD] batch block reads/writes during `AutoFile` serialization"
   }
  ],
  "conflicts": []
 },
 "stack": {
  "shares_commits_with": [],
  "based_on": [],
  "base_for": []
 },
 "review_paths": [],
 "body": "**Problem:** Block and undo writers buffer serialized data, then rely on `BufferedWriter`'s destructor to write the remaining bytes.\n`fwrite()` can report fewer bytes written than requested after a local storage error, e.g. if the filesystem fills up while a block is being written (my 1 TB benchmarking servers have recently been filling up, so I've been seeing these failures more often).\n`AutoFile::write_buffer()` converts that result into an exception and because the destructor is implicitly non-throwing, the exception invokes `std::terminate` instead of reaching normal storage-error handling.\nCalling `flush()` before destruction is insufficient on its own because a failed flush leaves the same bytes pending, so the destructor retries the write while the original exception unwinds.\n\nI introduced this bug in #31551 when adding the `BufferedWriter`.\n\n**Fix:** `BufferedWriter` now clears the pending byte count before writing, preventing a destructor retry after a failed explicit flush.\nBlock and undo writers flush before destruction, so write failures follow normal storage-error handling instead of terminating the process.\n\nManual short-write reproducer\n\n```sh\nsed -i '' '122s/src.size()/0/' src/streams.cpp\ncmake -B build && cmake --build build -j && build/bin/bitcoind -regtest -datadir=\"$(mktemp -d)\"; echo \"exit=$?\"\n```\n\nWith implicit flushing, `bitcoind` aborts:\n[quoted text omitted]\n  zsh: abort      build/bin/bitcoind -regtest -datadir=\"$(mktemp -d)\"\n  exit=134\n\nWith explicit flushing, it reports `Failed to write genesis block` and exits 1 instead of terminating the process:\n[quoted text omitted]\nexit=1",
 "commits": [
  {
   "sha": "2a5d83f5784cbeae9808e96193a1a516bea398f0",
   "date": "2026-08-20T21:40:04Z",
   "message": "test: characterize failed buffered flush\n\nA failed explicit `BufferedWriter` flush leaves bytes pending, so the destructor retries the write while the exception unwinds.\n\nRecord both writes while allowing the retry to succeed. This establishes the behavior that must change before callers can safely flush before destruction."
  },
  {
   "sha": "cd8ea7ebdc88304dc5ffd8b6343879c1ae94edde",
   "date": "2026-08-20T21:43:54Z",
   "message": "streams: do not retry failed flushes\n\nA failed `BufferedWriter::flush()` leaves bytes pending, so the destructor retries the write while the exception unwinds.\n\nClear the pending byte count before writing to the underlying stream. The original exception can then propagate without another write."
  },
  {
   "sha": "d5f424103bfb51a5aae3f91b023f1c23742e16d0",
   "date": "2026-08-20T21:43:54Z",
   "message": "blockstorage: handle buffered write failures\n\n`BufferedWriter`'s destructor is implicitly non-throwing, so a failure while it writes pending block or undo bytes invokes `std::terminate` before normal exception handling can report it.\n\nFlush block and undo writers before destruction. Write failures can then follow the existing error paths and produce a controlled storage failure."
  }
 ],
 "timeline": [
  {
   "t": "2026-08-20T22:21:58Z",
   "kind": "comment",
   "who": "andrewtoth",
   "assoc": "MEMBER",
   "text": "[quoted text omitted]\n\nCan you expand on why this is a problem? If there is a storage error, don't we have to crash anyways?"
  },
  {
   "t": "2026-08-20T22:35:20Z",
   "kind": "comment",
   "who": "l0rinc",
   "assoc": "MEMBER",
   "text": "[quoted text omitted]\n\nYes, the node still has to stop, but `std::terminate` bypasses the existing error handling.\n\n[quoted text omitted]\nHandling the exception lets the node report the storage error and run its normal shutdown cleanup."
  },
  {
   "t": "2026-08-21T11:56:41Z",
   "kind": "comment",
   "who": "maflcko",
   "assoc": "MEMBER",
   "text": "This reminds me of the other \"exceptions can terminate\" topics.\n\nIt would be good to find a code pattern that avoids this class of problem wholesale. It doesn't seem a great use of human (or LLM) review time to spend on such issues one-by-one. If we care about those issues, it would be better if the compiler or a clang-tidy analysis could find and prevent all of them."
  },
  {
   "t": "2026-09-05T20:25:47Z",
   "kind": "comment",
   "who": "w0xlt",
   "assoc": "CONTRIBUTOR",
   "text": "Concept ACK\n\nI ran into the same bug while working in this area and implemented an alternative fix before finding this PR.\n\nMy version removes writes from the `BufferedWriter` destructor and requires explicit final flushing. It also adjusts `AutoFile` cleanup during exception unwinding and adds an exception handler for undo-write failures.\n\nhttps://github.com/w0xlt/bitcoin/tree/streams/buffered-write-errors\n\nFeel free to reuse anything you find useful."
  }
 ],
 "labels_log": [],
 "state_log": [],
 "text_chars": 3877,
 "text_tokens_estimate": 969,
 "changed_paths": [
  "src/node/blockstorage.cpp",
  "src/streams.h",
  "src/test/streams_tests.cpp"
 ],
 "files": [
  {
   "path": "src/node/blockstorage.cpp",
   "add": 2,
   "del": 1
  },
  {
   "path": "src/streams.h",
   "add": 4,
   "del": 2
  },
  {
   "path": "src/test/streams_tests.cpp",
   "add": 14,
   "del": 0
  }
 ],
 "test_lines": 14,
 "git": {
  "head": "d5f424103bfb51a5aae3f91b023f1c23742e16d0",
  "head_matches_backup": true,
  "base": "bf8402c8803f085a50df96cb7956033cd252e9ab",
  "commits": [
   {
    "sha": "2a5d83f578",
    "subject": "test: characterize failed buffered flush",
    "files": 1,
    "add": 14,
    "del": 0
   },
   {
    "sha": "cd8ea7ebdc",
    "subject": "streams: do not retry failed flushes",
    "files": 2,
    "add": 5,
    "del": 3
   },
   {
    "sha": "d5f424103b",
    "subject": "blockstorage: handle buffered write failures",
    "files": 1,
    "add": 2,
    "del": 1
   }
  ],
  "patch_truncated": false
 },
 "input_hash": "c220a7ad70449370",
 "extracted_at": "2026-09-17T16:15:31+00:00"
}