{
 "number": 36521,
 "repo": "bitcoin/bitcoin",
 "url": "https://github.com/bitcoin/bitcoin/pull/36521",
 "title": "Keep witness stacks in serialized form in CScriptWitness",
 "author": "sipa",
 "author_association": "MEMBER",
 "created_at": "2026-10-14T06:23:27Z",
 "updated_at": "2026-10-31T02:21:00Z",
 "description_hash": "abe2e28973ffb262",
 "age_days": 18,
 "draft": false,
 "labels": [
  "Needs rebase"
 ],
 "milestone": null,
 "base": "master",
 "head_sha": "bd512269956afe1baa692d9893e68500356a859e",
 "head_ref": "202610_witpack",
 "head_repo": "sipa/bitcoin",
 "head_history": [
  {
   "t": "2026-10-14T06:26:41Z",
   "sha": "5d2840550be7f083fa84f39b886d857bfad236c8"
  },
  {
   "t": "2026-10-14T07:16:54Z",
   "sha": "091f56b0c4ceb9138a91b8bdcea71951d77dea10"
  },
  {
   "t": "2026-10-14T07:25:40Z",
   "sha": "e1cbb976d2a7545020e7683c6228dd1dc3424f2a"
  },
  {
   "t": "2026-10-19T23:08:35Z",
   "sha": "d3fb8e83c65220cf33f32257cf108f1186319d1a"
  },
  {
   "t": "2026-10-22T18:24:33Z",
   "sha": "bd512269956afe1baa692d9893e68500356a859e"
  }
 ],
 "additions": 1254,
 "deletions": 279,
 "changed_files": 76,
 "commit_count": 17,
 "size_bucket": "XL",
 "mergeable_state": "dirty",
 "bot": {
  "drahtbot": {
   "present": true,
   "reviews": {
    "concept_nack": [
     {
      "login": "purpleKarrot",
      "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6503407615"
     }
    ],
    "concept_ack": [
     {
      "login": "furszy",
      "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6162446479"
     },
     {
      "login": "0xB10C",
      "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6162449198"
     },
     {
      "login": "instagibbs",
      "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6181345136"
     },
     {
      "login": "w0xlt",
      "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6185721229"
     },
     {
      "login": "rkrux",
      "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6192784207"
     }
    ],
    "stale_ack": [
     {
      "login": "Yudis-bit",
      "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6282408300"
     }
    ]
   },
   "conflicts": []
  }
 },
 "acks_parsed": {
  "0xB10C": {
   "kind": "concept_ack",
   "hash": null,
   "t": "2026-10-14T06:35:30Z",
   "stale": false
  },
  "w0xlt": {
   "kind": "concept_ack",
   "hash": null,
   "t": "2026-10-15T06:10:17Z",
   "stale": false
  },
  "rkrux": {
   "kind": "concept_ack",
   "hash": null,
   "t": "2026-10-15T13:06:41Z",
   "stale": false
  },
  "Yudis-bit": {
   "kind": "ack",
   "hash": "d3fb8e8",
   "t": "2026-10-20T03:08:03Z",
   "stale": true
  }
 },
 "acks_tally": {
  "ack": 0,
  "stale_ack": 1,
  "concept_ack": 3,
  "approach_ack": 0,
  "nack": 0,
  "concept_nack": 0,
  "approach_nack": 0
 },
 "reviews": {
  "approved": 0,
  "changes_requested": 0,
  "distinct_reviewers": [
   "0xB10C",
   "Yudis-bit",
   "fametrano",
   "furszy",
   "instagibbs",
   "purpleKarrot",
   "rkrux",
   "w0xlt"
  ]
 },
 "signals": {
  "needs_rebase": true,
  "ci_failed": false,
  "mergeable_state": "dirty",
  "last_author_activity": "2026-10-22T18:52:58Z",
  "last_reviewer_activity": "2026-10-29T17:13:16Z",
  "last_reviewer": "purpleKarrot",
  "author_silent_days": 10,
  "waiting_on_author_days": 3,
  "days_since_update": 1
 },
 "refs": {
  "mentioned": [
   35569
  ],
  "depends_on": [],
  "fixes": [],
  "linked_issues": [],
  "references": [
   {
    "number": 35569,
    "type": "pull",
    "state": "open",
    "merged": false,
    "merged_at": null,
    "title": "Encapsulation for CTransaction"
   }
  ],
  "conflicts": []
 },
 "stack": {
  "shares_commits_with": [],
  "based_on": [],
  "base_for": []
 },
 "review_paths": [],
 "body": "`CScriptWitness` stores a witness stack as a `std::vector<std::vector<unsigned char>>`, which costs a 24-byte vector object and an allocation for every element. This makes it possible to construct transactions that take a large amount of memory after deserialization, while becoming very slow to deserialize.\n\nThis PR stores the stack as its serialization in a single allocation (or none if the stack is empty), with a small header holding its size and the offset of the last element. Witnesses can no longer be modified in place, only allowing replacing them entirely. Access to the individual witness stack elements is done by on-the-fly deserialization instead. `CScriptWitness` shrinks from 24 to 8 bytes, and `CTxIn` from 112 to 96 bytes.\n\nLater commits also introduce stricter checking during deserialization, rejecting the following before fully deserializing:\n* Transactions with more inputs/outputs than are possible in a valid transaction.\n* Transactions with 0 inputs or 0 outputs (applied in P2P, but not RPC etc. for backward compatibility).\n* Transactions/blocks that exceed the 4M weight limit (also only applied in P2P).\n\nThe result is a significant reduction in the worst-case time and memory allocations needed for deserialization, both in valid and invalid transactions. With it, the worst-case memory of a valid deserialized block drops from 106.8 MiB to 6.0 MiB. Memory usage during deserialization can temporarily be higher: up to 7.8 MiB for valid blocks, and up to 13.4 MiB for invalid blocks, which can combine a few temporary vector allocations of up to 5 MB each.\n\nDeserialization time and memory of the deserialized block, master \u2192 PR. Runtimes are medians in milliseconds. Memory usage is in MiB, and \u2715 means rejected at deserialization time. Peak memory refers to the memory during deserialization itself, not persisted in the deserialized `CBlock` object. Numbers over 50 ms, 20 MiB usage, or 100 MiB peak are in **bold**.\n\n| block | gcc | clang | mem | peak mem |\n|---|--:|--:|--:|--:|\n| **Valid blocks** |\n| 1 tx, max 1-byte wit elems | **99.1** \u2192 16.0 | **111.4** \u2192 21.9 | **106.8** \u2192 3.8 | **145.9** \u2192 7.8 |\n| 1 tx, max empty wit elems | **279.2** \u2192 18.7 | **293.7** \u2192 25.1 | **91.5** \u2192 3.8 | **182.2** \u2192 7.8 |\n| max txs, 1 txin, 1 txout, 1-item wit | 12.33 \u2192 13.39 | 15.00 \u2192 15.43 | 6.7 \u2192 6.0 | 6.7 \u2192 6.0 |\n| max txs, 1 txin and 1 txout | 6.43 \u2192 7.04 | 7.87 \u2192 8.30 | 5.8 \u2192 5.6 | 5.8 \u2192 5.6 |\n| 1 tx, max txins, 1-elem wit | 4.53 \u2192 4.68 | 5.04 \u2192 5.37 | 4.0 \u2192 2.9 | 4.0 \u2192 2.9 |\n| mainnet block 969729 | 5.30 \u2192 5.47 | 5.97 \u2192 6.20 | 3.5 \u2192 2.9 | 3.5 \u2192 2.9 |\n| **Invalid blocks** |\n| 400k txs, many wit elems, truncated | **84.8** \u2192 16.1 | **95.6** \u2192 21.8 | \u2715 \u2192 \u2715 | **150.7** \u2192 12.6 |\n| 400k txs, big wit+scriptsig, truncated | **84.0** \u2192 16.1 | **96.0** \u2192 21.7 | \u2715 \u2192 \u2715 | **150.7** \u2192 13.4 |\n| 1 tx, 2M 1-byte wit elems | **98.0** \u2192 10.6 | **109.9** \u2192 16.2 | **106.8** \u2192 \u2715 | **145.9** \u2192 7.8 |\n| 1 tx, 4M empty wit elems | **271.0** \u2192 13.4 | **288.9** \u2192 19.6 | **91.6** \u2192 \u2715 | **182.2** \u2192 7.8 |\n| max empty txs | **101.0** \u2192 0.006 | **139.0** \u2192 0.008 | **67.1** \u2192 \u2715 | 67.1 \u2192 4.8 |\n| max txs, 1 txin, 1 txout | 27.6 \u2192 7.3 | 34.5 \u2192 8.5 | **23.4** \u2192 \u2715 | 23.4 \u2192 6.4 |\n\nSo in addition to improving worst cases very significantly, it also provides modest memory usage improvements to the common case. This translates to mildly increasing the effective capacity of memory-bounded data structures that store transactions, like the mempool.\n\nSome notable changes:\n* By implementing some checks (for inherently consensus-invalid things) in the deserialization logic, behavior may change. In particular, deserialization errors in P2P cause messages to be dropped/ignored, rather than triggering validation or net processing logic (which may include disconnections and/or inserting into reject filters).\n* The rejection of transactions with too many inputs or outputs applies everywhere, including in RPC. That's hopefully a desirable change, but a change in behavior still.\n* Random access to the witness stack elements becomes $\\mathcal{O}(n)$. This is not a problem for any validation logic, but may be relevant for the libbitcoinkernel API which exposes an individual element accessor.\n\nIssue reported by furszy and 0xB10C.\n\nThere are a few possible follow-ups:\n* Enforce the 400k WU policy limit during transaction deserialization when active (instead of the 4M WU consensus limit).\n* Extend this approach to entire `CTransaction`, allowing all transaction data to be stored packed together in a few allocations (rather than one per input, one per output, one per witness) plus some index structure to access it efficiently. That's a much bigger change, needing abstractions like #35569, but would reduce the maximum memory usage per WU for valid blocks from around 1.5 bytes/WU to 1.1 bytes/WU (compared to ~28 bytes/WU in master).",
 "commits": [
  {
   "sha": "d10188e66a66dcc6ec81a6e13ce887217d8678ac",
   "date": "2026-10-22T14:16:57Z",
   "message": "bench: add block deserialization benchmarks for synthetic blocks\n\nAdd benchmarks deserializing blocks filled up to the weight limit with\nsynthetic transactions of a few shapes that stress different parts of\ndeserialization:\n* spends of P2WPKH outputs to P2WPKH outputs (a typical segwit block)\n* the smallest segwit transactions\n* one transaction with as many inputs as fit,\n* one transaction with as many outputs as fit,\n* one transaction whose witness has as many 1-byte elements as fit."
  },
  {
   "sha": "e41dd47374321236edb7423b703d59fdc8c7a55f",
   "date": "2026-10-22T14:16:57Z",
   "message": "script: give CScriptWitness an interface that hides its representation\n\nAdd element access through spans (forward iteration, front(), back(), and\nindexing), conversion to and construction from a vector of elements,\nserialization, and memory usage accounting, so that users no longer need to\naccess the stack vector directly. This prepares for a future commit which\nwill replace the representation with a more compact & efficient one.\n\nThe stack vector itself stays public until the next commits have converted\nits users."
  },
  {
   "sha": "111a1d381d320cc5dc0249670dac6a9ed01e2608",
   "date": "2026-10-22T14:16:57Z",
   "message": "test: check CScriptWitness against the vector of vectors it represents\n\nAdd a unit test and a fuzz test that compare CScriptWitness's interface\n(accessors, construction, and serialization) with the\nstd::vector<std::vector<unsigned char>> it represents. The unit test\nfocuses on element counts and sizes around CompactSize encoding\nboundaries."
  },
  {
   "sha": "67d5ec76ffa997d33a520074f4d16186b3c1cc66",
   "date": "2026-10-22T14:20:58Z",
   "message": "scripted-diff: use CScriptWitness's interface instead of its stack\n\nConvert the uses of CScriptWitness::stack for which the new interface\nis a simple substitution. The remaining ones are converted manually in\nthe next commits.\n\n-BEGIN VERIFY SCRIPT-\nFILES=$(git grep -l -E '\\.stack\\b|(scriptWitness|final_script_witness)\\.SetNull\\(\\)' -- src ':!src/script/miniscript.*' ':!src/script/script.h')\n\n# Element count, emptiness, and last element: the witness has size(), empty() and back() itself. Lines that need the\n# stack itself (to compute the size of its serialization without the element count, to pass the last element where a\n# vector is expected, or to index all elements in a loop) are left for the manual changes.\nsed -i -E -e '/GetSizeOfCompactSize|TAPROOT_LEAF_TAPSCRIPT|i < txin\\.scriptWitness/b' \\\n    -e 's/\\.stack\\.(size|empty|back)\\(\\)/.\\1()/g' $FILES\n\n# Reading an element by index: the witness has operator[] itself. Writes to elements, and loops indexing all elements\n# (which can be quadratic with indexing that is not constant-time), are left for the manual changes.\nsed -i -E -e '/MAX_STANDARD_P2WSH_STACK_ITEM_SIZE|HexStr\\(txin\\.scriptWitness|\\.stack\\[[^]]*\\]( =|\\.resize)|^\\s*\\+\\+/b' \\\n    -e 's/\\.stack\\[/[/g' $FILES\n\n# Const references to the stack: refer to the witness itself, which provides the same read-only accessors.\nsed -i -E 's/^(\\s*const auto& \\w+( = |\\{).*)\\.stack(\\}?;)$/\\1\\3/' $FILES\n\n# Comparisons of two stacks: compare the witnesses, which are equal exactly when their stacks are.\nsed -i -E '/\\.stack [!=]= /s/\\.stack\\b//g' $FILES\n\n# Serialization of the stack, and its size: serialize the witness itself, which has the same serialization.\nsed -i -E -e 's/ (>>|<<) ([][A-Za-z0-9_.]+)\\.stack( <<|;)/ \\1 \\2\\3/' \\\n    -e 's/(SerializeToVector|UnserializeFromVector)\\(s, (\\w+)\\.stack\\)/\\1(s, \\2)/' \\\n    -e 's/::GetSerializeSize\\(([][A-Za-z0-9_.]+)\\.stack\\)/::GetSerializeSize(\\1)/' $FILES\n\n# Range-for loops over the stack: iterate over the witness. Its elements are spans, returned by value, so they are not\n# bound by reference.\nsed -i -E 's/for \\(const auto& (\\w+) : ([][A-Za-z0-9_.]+)\\.stack\\)/for (const auto \\1 : \\2)/' $FILES\n\n# Clearing the stack (SetNull(), clear() or resize(0)): assign an empty witness. A clear() followed by pushing elements\n# in a loop is left for the manual changes.\nsed -i -E -e '/txin\\.scriptWitness\\.stack\\.clear/b' \\\n    -e 's/(scriptWitness|final_script_witness)\\.SetNull\\(\\)/\\1 = {}/' \\\n    -e 's/(scriptWitness)\\.stack\\.(clear\\(\\)|resize\\(0\\));$/\\1 = {};/' $FILES\n\n# Assigning a vector of elements to the stack: assign a witness constructed from it (moving from the vector is\n# pointless, as the witness is constructed from a const reference).\nsed -i -E 's/(\\w+)\\.stack = (std::move\\((\\w+)\\)|(.*));$/\\1 = CScriptWitness{\\3\\4};/' $FILES\n\n# Pushing a single element onto an empty witness (in these benchmarks and tests): assign a witness constructed from\n# that element. The one place that pushes onto a non-empty witness (to make a different witness) is left for the\n# manual changes, as are pushes in loops.\nsed -i -E '/ tx\\.vin\\[0\\]\\.scriptWitness\\.stack\\.push_back\\(\\{5\\}\\)|Consume/!s/(\\w+)\\.stack\\.push_back\\((.*)\\);$/\\1 = CScriptWitness{{\\2}};/' \\\n    src/bench/block_assemble.cpp src/bench/blockencodings.cpp src/bench/mempool_eviction.cpp src/bench/mempool_stress.cpp src/bench/rpc_mempool.cpp \\\n    src/test/fuzz/connect_block.cpp src/test/fuzz/txdownloadman.cpp src/test/orphanage_tests.cpp src/test/txdownload_tests.cpp src/test/validation_block_tests.cpp\n\n# IsNull(): use empty(), which the witness provides along with size() and its other container-like accessors.\nsed -i -E -e 's/(scriptWitness|final_script_witness)\\.IsNull\\(\\)/\\1.empty()/g' -e 's/witness->IsNull\\(\\)/witness->empty()/g' \\\n    $(git grep -l -E '(scriptWitness|final_script_witness)\\.IsNull\\(\\)|witness->IsNull\\(\\)' -- src)\n-END VERIFY SCRIPT-"
  },
  {
   "sha": "376dbf806914408b01fea2b2eb04e56805c2ac48",
   "date": "2026-10-22T14:20:59Z",
   "message": "script: let ExecuteWitnessScript take its initial stack by value\n\nThis lets callers move the stack into it, rather than having it be\ncopied from a span. This will be used in a future commit."
  },
  {
   "sha": "0b4f24be6f40100cf40048115d189be796464291",
   "date": "2026-10-22T14:20:59Z",
   "message": "refactor: use CScriptWitness's interface where that needs more than a rename\n\nConvert the remaining non-test uses of CScriptWitness::stack, which need\nmore than a substitution. The script interpreter now converts the\nwitness stack to a vector of vectors once, and moves the remaining\nelements into the execution stack, rather than copying them."
  },
  {
   "sha": "9e22357e32a815cb3a0bccb977fe116bfa121d7a",
   "date": "2026-10-22T14:20:59Z",
   "message": "test, bench, fuzz: build witness stacks before constructing a CScriptWitness\n\nConvert the remaining uses of CScriptWitness::stack in tests, benchmarks,\nand fuzz tests, by constructing witnesses from a separately-built vector\nof elements, rather than modifying the stack in place."
  },
  {
   "sha": "74b1e69684021d23819ab7169cd0929d865096d3",
   "date": "2026-10-22T14:20:59Z",
   "message": "streams: add SpanReader::Remaining()\n\nThis returns the data that has not been read yet, as a span referring\nto the underlying data. It will be used in the next commit to find the\nelements of a witness stack within its serialization."
  },
  {
   "sha": "6a5eeda000452b31628afec8ee848dc4fe617aaa",
   "date": "2026-10-22T14:20:59Z",
   "message": "script: store witness stacks as their serialization\n\nReplace CScriptWitness's vector of vectors with a single allocation,\ncontaining the size of the stack's serialization (4 bytes, little\nendian), followed by that serialization. An empty stack uses no\nallocation at all.\n\nThis means a witness stack no longer needs one allocation per non-empty\nelement plus one for the outer vector, and no longer has a 24-byte\noverhead per element. A block whose witnesses consist of millions of\ntiny elements thus no longer costs tens of bytes of memory and an\nallocation per byte. It also shrinks CScriptWitness from 24 to 8 bytes,\nand thus CTxIn from 112 to 96 bytes. The downside is that indexing\nbecomes O(n) in the distance from the bottom of the stack.\n\nThis is the simplest implementation of this representation:\ndeserialization still goes through a vector of vectors, and elements\nare found by parsing the serialization with SpanReader and\nReadCompactSize. Future commits will make both faster."
  },
  {
   "sha": "a96bec251790a2b83e4da7a9c642576366849344",
   "date": "2026-10-22T14:20:59Z",
   "message": "script: deserialize witness stacks directly into their serialization\n\nInstead of deserializing a witness stack as a vector of vectors and then\nserializing that again, build the serialization directly while reading,\nwith the same checks. This makes deserializing a block whose witness\nconsists of millions of 1-byte elements about 10x faster.\n\nTo support this, add AppendCompactSize to serialize.h, which cheaply\nappends a CompactSize to a byte vector."
  },
  {
   "sha": "0871946014794292a85a2159fcd070d10fdf7e9c",
   "date": "2026-10-22T14:20:59Z",
   "message": "script: decode the CompactSizes in witness stacks inline\n\nAdd DecodeCompactSize to serialize.h, which decodes a CompactSize from a\nspan without any checks, making it small enough to be inlined.\n\nUse it to find the elements in a CScriptWitness's serialization, which\nis known to be valid as CScriptWitness constructed it. This makes\niterating over witness stacks 5-7x faster, both for real blocks and for\nblocks whose witnesses consist of millions of 1-byte elements. As a\nbonus, script.h no longer needs streams.h."
  },
  {
   "sha": "dc91fd9364c4c706977bea31056a8065290c193f",
   "date": "2026-10-22T14:20:59Z",
   "message": "script: store the offset of a witness stack's last element\n\nAdd the offset of the last element to the header of CScriptWitness's\nallocation, making back() and accessing the last element by index O(1).\nThe last element is used in P2WSH sigop counting, policy checks, and\nBIP352 public key extraction.\n\nThis adds 4 bytes to the allocation of every non-empty witness stack,\nbut does not change the minimum allocation size of a single-element\nstack, nor the worst-case ratio of memory usage to weight or size."
  },
  {
   "sha": "dbc10d44f955ef27ecf498482bc547ae0f188a34",
   "date": "2026-10-22T15:11:18Z",
   "message": "primitives: reject txs with more inputs or outputs than fit in a block\n\nWhen deserializing a transaction, read the input and output counts\nfirst, and fail before reading the inputs or outputs if more of them\nare claimed than can fit in a block, given their minimum size (41 bytes\nfor an input, 9 for an output). Such a transaction can never be valid.\n\nThis allows us to allocate the vin and vout vectors immediately, rather\nthan sequentially in batches of 5 MB as the std::vector deserializer\ndoes, because that memory becomes bounded: to 2.3 MB for 24390 inputs,\nand 5.3 MB for 111111 outputs. It is a preparation for further changes\nthat rely on this structuring of conditionals.\n\nThis affects all places where transactions are deserialized. Through\nRPC, such transactions can no longer be decoded. From P2P, a transaction\nor block containing one is ignored like other malformed messages, rather\nthan rejected by validation."
  },
  {
   "sha": "476a8ba657693f1ebefdfe46ebcfca5f068650af",
   "date": "2026-10-22T15:43:42Z",
   "message": "p2p: reject transactions without inputs or outputs when deserializing\n\nAdd a reject_empty flag to TransactionSerParams, which makes\ndeserialization fail as soon as a transaction is found to have no\ninputs or no outputs. Such transactions are never valid, but are\ncurrently fully deserialized (along with the rest of the message) before\nvalidation rejects them.\n\nThis reduces the memory a peer can make us allocate by sending a block.\nA block full of 10-byte empty transactions was the previous worst case;\nwith this change, the worst case becomes 1-input 1-output transactions\nwith a significantly lower per-WU memory usage.\n\nThe flag is enabled (as TX_WITH_WITNESS_P2P) only where transactions are\nreceived from P2P peers: tx and block messages, and the transactions in\ncmpctblock and blocktxn messages. Like other malformed messages, these\nare now ignored, rather than, for a block, causing the peer to be\ndisconnected, or, for a transaction, being added to the reject filters.\nThe functional tests are updated accordingly."
  },
  {
   "sha": "8072be3acb1e8b9f1ae5b195ff4d7d32ef203234",
   "date": "2026-10-22T15:43:42Z",
   "message": "policy: iterate over witness stacks directly in IsWitnessStandard\n\nInstead of converting a witness stack to a vector of spans for the P2WSH\nand Taproot checks, use its size() and back() (which are constant-time)\nand iterate over its elements directly. This avoids an allocation for\nevery witness spend that is checked."
  },
  {
   "sha": "85495b700de967c7ebf1fa756b2f270481fd0ea1",
   "date": "2026-10-22T15:43:42Z",
   "message": "refactor: use the stream's parameters for compact block transactions\n\nBlockTransactions and PrefilledTransaction (de)serialized their\ntransactions with TX_WITH_WITNESS_P2P, overriding any parameters of the\nstream. Use the stream's parameters instead, like CBlock does, and pass\nTX_WITH_WITNESS where they are serialized and TX_WITH_WITNESS_P2P where\nthey are deserialized. This does not change behavior (serialization only\ndepends on allow_witness), but makes it possible to deserialize them\nwith other parameters."
  },
  {
   "sha": "bd512269956afe1baa692d9893e68500356a859e",
   "date": "2026-10-22T17:55:32Z",
   "message": "p2p: limit the weight of transactions and blocks while deserializing\n\nAdd an optional weight budget to TransactionSerParams. When it is set,\nthe budget is decremented while transactions are being deserialized\n(charging the minimum weight of inputs and outputs before allocating\nmemory for them), causing failure as soon as the budget is exceeded.\n\nUse this for tx, block, cmpctblock and blocktxn messages, with a budget\nof MAX_BLOCK_WEIGHT minus the header: the transactions in a block,\ncmpctblock or blocktxn message are part of one block (for blocktxn, this\ndoes not account for those of the block that are already known), and a\ntransaction that does not fit in a block cannot be valid.\n\nThe P2P message size limit allows blocks of up to 4 MB, which is up to\n16M weight, all of which is currently deserialized before validation\nrejects it (unless a transaction has more inputs or outputs than fit in\na block). With this change, the weight deserialized from a message is\nat most MAX_BLOCK_WEIGHT minus the header.\n\nMessages exceeding the budget are ignored, like other malformed\nmessages, rather than causing the peer to be disconnected for sending an\ninvalid block."
  }
 ],
 "timeline": [
  {
   "t": "2026-10-14T06:26:41Z",
   "kind": "force_push",
   "who": "sipa",
   "commit": "5d2840550be7f083fa84f39b886d857bfad236c8",
   "id": "push:5d2840550be7f083fa84f39b886d857bfad236c8"
  },
  {
   "t": "2026-10-14T06:35:18Z",
   "kind": "comment",
   "who": "furszy",
   "assoc": "MEMBER",
   "text": "Obvious concept ACK",
   "id": "c:6162446479",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6162446479",
   "hash": "d9bb80116c399f91",
   "edited": null
  },
  {
   "t": "2026-10-14T06:35:30Z",
   "kind": "comment",
   "who": "0xB10C",
   "assoc": "MEMBER",
   "text": "Concept ACK",
   "id": "c:6162449198",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6162449198",
   "hash": "e8334cc43879b73c",
   "edited": null
  },
  {
   "t": "2026-10-14T07:16:54Z",
   "kind": "force_push",
   "who": "sipa",
   "commit": "091f56b0c4ceb9138a91b8bdcea71951d77dea10",
   "id": "push:091f56b0c4ceb9138a91b8bdcea71951d77dea10"
  },
  {
   "t": "2026-10-14T07:25:40Z",
   "kind": "force_push",
   "who": "sipa",
   "commit": "e1cbb976d2a7545020e7683c6228dd1dc3424f2a",
   "id": "push:e1cbb976d2a7545020e7683c6228dd1dc3424f2a"
  },
  {
   "t": "2026-10-15T00:13:48Z",
   "kind": "comment",
   "who": "instagibbs",
   "assoc": "MEMBER",
   "text": "definite concept ACK, this issue with exceptional tx types shows up in a lot of places, like https://github.com/bitcoin/bitcoin/pull/36015",
   "id": "c:6181345136",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6181345136",
   "hash": "656f02f38a640cd8",
   "edited": null
  },
  {
   "t": "2026-10-15T06:10:17Z",
   "kind": "comment",
   "who": "w0xlt",
   "assoc": "CONTRIBUTOR",
   "text": "Concept ACK",
   "id": "c:6185721229",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6185721229",
   "hash": "e8334cc43879b73c",
   "edited": null
  },
  {
   "t": "2026-10-15T13:06:41Z",
   "kind": "comment",
   "who": "rkrux",
   "assoc": "MEMBER",
   "text": "Concept ACK",
   "id": "c:6192784207",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6192784207",
   "hash": "e8334cc43879b73c",
   "edited": null
  },
  {
   "t": "2026-10-17T13:22:40Z",
   "kind": "comment",
   "who": "Yudis-bit",
   "assoc": "NONE",
   "text": "In `IsWitnessStandard` (`src/policy/policy.cpp`), `tx.vin[i].scriptWitness.ToSpans()` allocates on the heap for every Taproot input, even though for standard keypath spends (`size() == 1`) no policy rules apply. Since `back()` is O(1), checking `witness.size() >= 2` and `witness.back()` upfront before calling `ToSpans()` avoids that allocation.\n\nSimilarly in the P2WSH check, `ToSpans()` is used for indexed access up to `sizeWitnessStack`. Iterating directly over `scriptWitness` with a counter avoids allocating the vector of spans without doing O(n^2) indexing.",
   "id": "c:6234391822",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6234391822",
   "hash": "55749b85e63ea387",
   "edited": null
  },
  {
   "t": "2026-10-19T23:08:35Z",
   "kind": "force_push",
   "who": "sipa",
   "commit": "d3fb8e83c65220cf33f32257cf108f1186319d1a",
   "id": "push:d3fb8e83c65220cf33f32257cf108f1186319d1a"
  },
  {
   "t": "2026-10-20T01:01:53Z",
   "kind": "comment",
   "who": "sipa",
   "assoc": "MEMBER",
   "text": "@Yudis-bit Thanks, addressed in a separate commit at the end.",
   "id": "c:6280847772",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6280847772",
   "hash": "45461576382f0be4",
   "edited": null
  },
  {
   "t": "2026-10-20T03:08:03Z",
   "kind": "comment",
   "who": "Yudis-bit",
   "assoc": "NONE",
   "text": "Tested ACK d3fb8e8 on Windows.",
   "id": "c:6282408300",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6282408300",
   "hash": "7a61327810c73c0e",
   "edited": null
  },
  {
   "t": "2026-10-22T18:24:33Z",
   "kind": "force_push",
   "who": "sipa",
   "commit": "bd512269956afe1baa692d9893e68500356a859e",
   "id": "push:bd512269956afe1baa692d9893e68500356a859e"
  },
  {
   "t": "2026-10-22T18:52:58Z",
   "kind": "comment",
   "who": "sipa",
   "assoc": "MEMBER",
   "text": "I have added changes that also reject transactions/blocks early during deserialization which exceed the 4M WU limit. This reduces the worst-case memory and deserialization time for invalid transactions/blocks further.",
   "id": "c:6348727144",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6348727144",
   "hash": "5fa2e63d9b6ab5bb",
   "edited": "2026-10-22T18:53:06Z"
  },
  {
   "t": "2026-10-25T07:45:55Z",
   "kind": "comment",
   "who": "fametrano",
   "assoc": "NONE",
   "text": "`VerifyWitnessProgram` calls `witness.ToVectors()` before it looks at the witness version, so the valid \"1 tx, max empty wit elems\" block from the description still allocates about 91 MiB, now during script verification.\n\nSuch a witness is only valid when no script runs on it, as when it spends an unknown witness version, so the copy is never used. As in 8072be3acb, calling `ToVectors()` only in the branches that reach `ExecuteWitnessScript` would keep the saving through validation.\n\nI counted heap in a program that replaces `operator new`, linked against each build, and took times from regtest nodes running each build. With 3,990,000 empty elements spending a v2 output, deserialization peaks at 7.8 MiB on bd51226995 against 182.7 MiB on its base. `VerifyScript` then makes a 91.3 MiB copy that the base does not make. The `Verify 1 txins` bench line goes from 0.02-0.03 ms to 18-24 ms. The block is still accepted faster than on the base (0.04-0.05 s against 0.14-0.20 s).\n\nI also compared the two builds over RPC and P2P in one functional test (edge cases, test vectors, blocks at the weight limit, PSBTs), and on 400,000 mutated transactions fed to each deserializer. I checked decoding, wtxids, sighashes, weight and sigops against btclib, a parser I maintain. Apart from error messages, the only differences were those the description lists. I can share the code, corpus generator included, as a gist.",
   "id": "c:6395031310",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6395031310",
   "hash": "0332542a9f45fd5f",
   "edited": null
  },
  {
   "t": "2026-10-29T17:13:16Z",
   "kind": "comment",
   "who": "purpleKarrot",
   "assoc": "MEMBER",
   "text": "May I dare to NACK this? I understand that keeping the witness data in serialized format considerably improves performance, but as I wrote in https://github.com/bitcoin/bitcoin/pull/35569#issuecomment-6311323619, there sure are hundreds of possibilities for data layouts that may be even more optimized for fast data access.\n\nInstead pushing 17 commits touching over 1000 lines in 76 files, just to establish an implementation that is proven better but not proven best, should we maybe concentrate on the task that would allow us to make such a change localized in a single place? This would give us the possibility to easily experiment with different data layouts and lead to an even faster implementation?",
   "id": "c:6503407615",
   "url": "https://github.com/bitcoin/bitcoin/pull/36521#issuecomment-6503407615",
   "hash": "5b194c9a8a6c287a",
   "edited": null
  }
 ],
 "labels_log": [
  {
   "t": "2026-10-14T06:27:53Z",
   "action": "labeled",
   "label": "CI failed",
   "who": "DrahtBot"
  },
  {
   "t": "2026-10-14T08:54:21Z",
   "action": "unlabeled",
   "label": "CI failed",
   "who": "DrahtBot"
  },
  {
   "t": "2026-10-22T10:45:48Z",
   "action": "labeled",
   "label": "Needs rebase",
   "who": "DrahtBot"
  },
  {
   "t": "2026-10-22T20:35:21Z",
   "action": "unlabeled",
   "label": "Needs rebase",
   "who": "DrahtBot"
  },
  {
   "t": "2026-10-30T14:26:48Z",
   "action": "labeled",
   "label": "Needs rebase",
   "who": "DrahtBot"
  }
 ],
 "state_log": [],
 "text_chars": 20826,
 "text_tokens_estimate": 5206,
 "changed_paths": [
  "src/bench/CMakeLists.txt",
  "src/bench/block_assemble.cpp",
  "src/bench/block_deserialize.cpp",
  "src/bench/blockencodings.cpp",
  "src/bench/mempool_eviction.cpp",
  "src/bench/mempool_stress.cpp",
  "src/bench/rpc_mempool.cpp",
  "src/bench/verify_script.cpp",
  "src/bitcoin-tx.cpp",
  "src/blockencodings.cpp",
  "src/blockencodings.h",
  "src/common/bip352.cpp",
  "src/consensus/validation.h",
  "src/core_io.cpp",
  "src/core_memusage.h",
  "src/kernel/bitcoinkernel.cpp",
  "src/net_processing.cpp",
  "src/node/miner.cpp",
  "src/policy/policy.cpp",
  "src/primitives/transaction.cpp",
  "src/primitives/transaction.h",
  "src/psbt.cpp",
  "src/psbt.h",
  "src/rpc/blockchain.cpp",
  "src/rpc/rawtransaction.cpp",
  "src/rpc/rawtransaction_util.cpp",
  "src/script/interpreter.cpp",
  "src/script/script.cpp",
  "src/script/script.h",
  "src/script/sign.cpp",
  "src/serialize.h",
  "src/signet.cpp",
  "src/streams.h",
  "src/test/bip352_tests.cpp",
  "src/test/blockencodings_tests.cpp",
  "src/test/fuzz/CMakeLists.txt",
  "src/test/fuzz/cmpctblock.cpp",
  "src/test/fuzz/connect_block.cpp",
  "src/test/fuzz/deserialize.cpp",
  "src/test/fuzz/miniscript.cpp",
  "src/test/fuzz/package_eval.cpp",
  "src/test/fuzz/script.cpp",
  "src/test/fuzz/script_assets_test_minimizer.cpp",
  "src/test/fuzz/script_witness.cpp",
  "src/test/fuzz/transaction.cpp",
  "src/test/fuzz/tx_pool.cpp",
  "src/test/fuzz/txdownloadman.cpp",
  "src/test/fuzz/txorphan.cpp",
  "src/test/fuzz/util.cpp",
  "src/test/miniscript_tests.cpp",
  "src/test/orphanage_tests.cpp",
  "src/test/private_broadcast_tests.cpp",
  "src/test/psbt_tests.cpp",
  "src/test/rbf_tests.cpp",
  "src/test/script_assets_tests.cpp",
  "src/test/script_tests.cpp",
  "src/test/serialize_tests.cpp",
  "src/test/sigopcount_tests.cpp",
  "src/test/streams_tests.cpp",
  "src/test/transaction_tests.cpp",
  "src/test/txdownload_tests.cpp",
  "src/test/txpackage_tests.cpp",
  "src/test/txvalidation_tests.cpp",
  "src/test/txvalidationcache_tests.cpp",
  "src/test/validation_block_tests.cpp",
  "src/test/validation_tests.cpp",
  "src/validation.cpp",
  "src/wallet/feebumper.cpp",
  "src/wallet/test/feebumper_tests.cpp",
  "src/wallet/test/fuzz/coincontrol.cpp",
  "src/wallet/wallet.cpp",
  "test/functional/data/invalid_txs.py",
  "test/functional/feature_block.py",
  "test/functional/mempool_accept.py",
  "test/functional/p2p_invalid_tx.py",
  "test/functional/p2p_segwit.py"
 ],
 "files": [
  {
   "path": "src/bench/CMakeLists.txt",
   "add": 1,
   "del": 0
  },
  {
   "path": "src/bench/block_assemble.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/bench/block_deserialize.cpp",
   "add": 155,
   "del": 0
  },
  {
   "path": "src/bench/blockencodings.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/bench/mempool_eviction.cpp",
   "add": 11,
   "del": 11
  },
  {
   "path": "src/bench/mempool_stress.cpp",
   "add": 2,
   "del": 2
  },
  {
   "path": "src/bench/rpc_mempool.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/bench/verify_script.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/bitcoin-tx.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/blockencodings.cpp",
   "add": 2,
   "del": 2
  },
  {
   "path": "src/blockencodings.h",
   "add": 2,
   "del": 2
  },
  {
   "path": "src/common/bip352.cpp",
   "add": 4,
   "del": 4
  },
  {
   "path": "src/consensus/validation.h",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/core_io.cpp",
   "add": 3,
   "del": 3
  },
  {
   "path": "src/core_memusage.h",
   "add": 2,
   "del": 5
  },
  {
   "path": "src/kernel/bitcoinkernel.cpp",
   "add": 5,
   "del": 3
  },
  {
   "path": "src/net_processing.cpp",
   "add": 25,
   "del": 10
  },
  {
   "path": "src/node/miner.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/policy/policy.cpp",
   "add": 10,
   "del": 9
  },
  {
   "path": "src/primitives/transaction.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/primitives/transaction.h",
   "add": 70,
   "del": 12
  },
  {
   "path": "src/psbt.cpp",
   "add": 6,
   "del": 6
  },
  {
   "path": "src/psbt.h",
   "add": 5,
   "del": 5
  },
  {
   "path": "src/rpc/blockchain.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/rpc/rawtransaction.cpp",
   "add": 5,
   "del": 5
  },
  {
   "path": "src/rpc/rawtransaction_util.cpp",
   "add": 3,
   "del": 2
  },
  {
   "path": "src/script/interpreter.cpp",
   "add": 25,
   "del": 16
  },
  {
   "path": "src/script/script.cpp",
   "add": 86,
   "del": 5
  },
  {
   "path": "src/script/script.h",
   "add": 184,
   "del": 8
  },
  {
   "path": "src/script/sign.cpp",
   "add": 6,
   "del": 6
  },
  {
   "path": "src/serialize.h",
   "add": 46,
   "del": 0
  },
  {
   "path": "src/signet.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/streams.h",
   "add": 2,
   "del": 0
  },
  {
   "path": "src/test/bip352_tests.cpp",
   "add": 6,
   "del": 8
  },
  {
   "path": "src/test/blockencodings_tests.cpp",
   "add": 49,
   "del": 12
  },
  {
   "path": "src/test/fuzz/CMakeLists.txt",
   "add": 1,
   "del": 0
  },
  {
   "path": "src/test/fuzz/cmpctblock.cpp",
   "add": 4,
   "del": 4
  },
  {
   "path": "src/test/fuzz/connect_block.cpp",
   "add": 5,
   "del": 4
  },
  {
   "path": "src/test/fuzz/deserialize.cpp",
   "add": 3,
   "del": 3
  },
  {
   "path": "src/test/fuzz/miniscript.cpp",
   "add": 13,
   "del": 11
  },
  {
   "path": "src/test/fuzz/package_eval.cpp",
   "add": 2,
   "del": 2
  },
  {
   "path": "src/test/fuzz/script.cpp",
   "add": 4,
   "del": 3
  },
  {
   "path": "src/test/fuzz/script_assets_test_minimizer.cpp",
   "add": 3,
   "del": 3
  },
  {
   "path": "src/test/fuzz/script_witness.cpp",
   "add": 92,
   "del": 0
  },
  {
   "path": "src/test/fuzz/transaction.cpp",
   "add": 17,
   "del": 0
  },
  {
   "path": "src/test/fuzz/tx_pool.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/test/fuzz/txdownloadman.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/test/fuzz/txorphan.cpp",
   "add": 2,
   "del": 3
  },
  {
   "path": "src/test/fuzz/util.cpp",
   "add": 5,
   "del": 4
  },
  {
   "path": "src/test/miniscript_tests.cpp",
   "add": 15,
   "del": 13
  },
  {
   "path": "src/test/orphanage_tests.cpp",
   "add": 4,
   "del": 2
  },
  {
   "path": "src/test/private_broadcast_tests.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/test/psbt_tests.cpp",
   "add": 2,
   "del": 2
  },
  {
   "path": "src/test/rbf_tests.cpp",
   "add": 1,
   "del": 2
  },
  {
   "path": "src/test/script_assets_tests.cpp",
   "add": 3,
   "del": 3
  },
  {
   "path": "src/test/script_tests.cpp",
   "add": 67,
   "del": 11
  },
  {
   "path": "src/test/serialize_tests.cpp",
   "add": 28,
   "del": 0
  },
  {
   "path": "src/test/sigopcount_tests.cpp",
   "add": 4,
   "del": 14
  },
  {
   "path": "src/test/streams_tests.cpp",
   "add": 29,
   "del": 0
  },
  {
   "path": "src/test/transaction_tests.cpp",
   "add": 166,
   "del": 3
  },
  {
   "path": "src/test/txdownload_tests.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/test/txpackage_tests.cpp",
   "add": 6,
   "del": 16
  },
  {
   "path": "src/test/txvalidation_tests.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/test/txvalidationcache_tests.cpp",
   "add": 5,
   "del": 3
  },
  {
   "path": "src/test/validation_block_tests.cpp",
   "add": 2,
   "del": 2
  },
  {
   "path": "src/test/validation_tests.cpp",
   "add": 7,
   "del": 7
  },
  {
   "path": "src/validation.cpp",
   "add": 3,
   "del": 3
  },
  {
   "path": "src/wallet/feebumper.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "src/wallet/test/feebumper_tests.cpp",
   "add": 4,
   "del": 1
  },
  {
   "path": "src/wallet/test/fuzz/coincontrol.cpp",
   "add": 2,
   "del": 2
  },
  {
   "path": "src/wallet/wallet.cpp",
   "add": 1,
   "del": 1
  },
  {
   "path": "test/functional/data/invalid_txs.py",
   "add": 6,
   "del": 0
  },
  {
   "path": "test/functional/feature_block.py",
   "add": 5,
   "del": 3
  },
  {
   "path": "test/functional/mempool_accept.py",
   "add": 9,
   "del": 0
  },
  {
   "path": "test/functional/p2p_invalid_tx.py",
   "add": 1,
   "del": 1
  },
  {
   "path": "test/functional/p2p_segwit.py",
   "add": 2,
   "del": 1
  }
 ],
 "test_lines": 917,
 "git": {
  "head": "bd512269956afe1baa692d9893e68500356a859e",
  "head_matches_backup": true,
  "base": "f655a8df755330911c6c109c8b1d04ef5dd4ea09",
  "patch_id": "5b1bb4606b8fdbaf",
  "commits": [
   {
    "sha": "d10188e66a",
    "subject": "bench: add block deserialization benchmarks for synthetic blocks",
    "files": 2,
    "add": 156,
    "del": 0
   },
   {
    "sha": "e41dd47374",
    "subject": "script: give CScriptWitness an interface that hides its representation",
    "files": 2,
    "add": 73,
    "del": 3
   },
   {
    "sha": "111a1d381d",
    "subject": "test: check CScriptWitness against the vector of vectors it represents",
    "files": 3,
    "add": 146,
    "del": 0
   },
   {
    "sha": "67d5ec76ff",
    "subject": "scripted-diff: use CScriptWitness's interface instead of its stack",
    "files": 43,
    "add": 100,
    "del": 100
   },
   {
    "sha": "376dbf8069",
    "subject": "script: let ExecuteWitnessScript take its initial stack by value",
    "files": 1,
    "add": 6,
    "del": 6
   },
   {
    "sha": "0b4f24be6f",
    "subject": "refactor: use CScriptWitness's interface where that needs more than a rename",
    "files": 7,
    "add": 36,
    "del": 23
   },
   {
    "sha": "9e22357e32",
    "subject": "test, bench, fuzz: build witness stacks before constructing a CScriptWitness",
    "files": 21,
    "add": 87,
    "del": 95
   },
   {
    "sha": "74b1e69684",
    "subject": "streams: add SpanReader::Remaining()",
    "files": 2,
    "add": 31,
    "del": 0
   },
   {
    "sha": "6a5eeda000",
    "subject": "script: store witness stacks as their serialization",
    "files": 3,
    "add": 201,
    "del": 47
   },
   {
    "sha": "a96bec2517",
    "subject": "script: deserialize witness stacks directly into their serialization",
    "files": 3,
    "add": 59,
    "del": 3
   },
   {
    "sha": "0871946014",
    "subject": "script: decode the CompactSizes in witness stacks inline",
    "files": 3,
    "add": 57,
    "del": 23
   },
   {
    "sha": "dc91fd9364",
    "subject": "script: store the offset of a witness stack's last element",
    "files": 2,
    "add": 45,
    "del": 27
   },
   {
    "sha": "dbc10d44f9",
    "subject": "primitives: reject txs with more inputs or outputs than fit in a block",
    "files": 3,
    "add": 56,
    "del": 10
   },
   {
    "sha": "476a8ba657",
    "subject": "p2p: reject transactions without inputs or outputs when deserializing",
    "files": 7,
    "add": 82,
    "del": 7
   },
   {
    "sha": "8072be3acb",
    "subject": "policy: iterate over witness stacks directly in IsWitnessStandard",
    "files": 1,
    "add": 6,
    "del": 9
   },
   {
    "sha": "85495b700d",
    "subject": "refactor: use the stream's parameters for compact block transactions",
    "files": 6,
    "add": 30,
    "del": 30
   },
   {
    "sha": "bd51226995",
    "subject": "p2p: limit the weight of transactions and blocks while deserializing",
    "files": 7,
    "add": 198,
    "del": 11
   }
  ],
  "patch_truncated": true
 },
 "patch_id": "5b1bb4606b8fdbaf",
 "input_hash": "f217fed2e715815b",
 "extracted_at": "2026-11-02T01:09:49+00:00"
}