<p>Old nodes never see the new serialisation format</p>
<p>When an old node asks a new node for data, the new node edits the data into the old serialisation format before sending it</p>

CC BY-SA 4.0
2023-09-10T18:05:37.913Z
Okay, but when a node just want to relay the transaction it got to all its peers, how will he know if the peer is old or new (updated) node?
dassd
CC BY-SA 4.0
2023-09-10T18:23:22.620Z
Part of the communications handshake distinguishes segwit nodes
RedGrittyBrick
CC BY-SA 4.0
2023-09-10T18:34:09.450Z
Okay, I understand they can do that when we're talking about relaying. But what about the block transaction received by the miner? How will the old nodes confirm this since they don't understand the new structure? I mean, it can't be the old serialization format without witnesses and markers/flags because then the old and new nodes would have a different blockchain and this block would have a different hash?
dassd
CC BY-SA 4.0
2023-09-10T18:35:25.647Z
@RedGrittyBrick While that's true, the handshake just lets the other side know that the peer can *offer* witness. To actually get witness data, a node has to ask for a witness_tx or a witness_block, rather than tx or block (if you ask for those, even from a segwit node, you get the stripped version).
Pieter Wuille
CC BY-SA 4.0
2023-09-10T18:36:41.250Z
@joke The txid of a transaction is independent of whether the witnesses are included or not, and the hash of a block is the hash of its header - which also doesn't include the witnesses. Witnesses data *is* committed to in blocks, but just by including the hash of all the witnesses in the coinbase transaction somewhere. So no, you can just drop all the witness data from a witness block, give it to a pre-segwit peer, and it will be valid to them.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T18:49:22.910Z
@PieterWuille So the stripped version (of both the block and the transaction) is always sent, with the fact that during handshaking, the segWit node somehow announces itself (it will set some flag or something) as segWit node and that can also send witness data?
dassd
CC BY-SA 4.0
2023-09-10T18:53:59.213Z
Right. NODE_WITNESS announces that witnesses are available, and if so, the `getdata` command can be used to ask for transactions, blocks, or for transactions+witness, or for block+witness. Without that flag, only transactions or blocks can be asked for.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T19:03:56.977Z
@PieterWuille Couple of question. 1) I assume that NODE_WITNESS flag is introduced with segWit, so it won't "confuse" old nodes? I mean, they will just ignore it and ask for `tx`/`block` (not `tx+witness`/`block+witness`)? 2) If we ask for `tx+witness`, we will get from node new serialization format (with marker+flag+witness), and if we ask for `tx` we get old serialization format? (of course all this in case that tx is segWit) 3) What does it mean `block+witness` and `block`? Is first that we get block consist of all tx with new ser format, while the second one is all tx with old ser format?
dassd
CC BY-SA 4.0
2023-09-10T19:08:32.157Z
@joke To go in this much depth, I suggest you just read BIP144 which specifies the protocol/serialization changes. BIP141 specifies the segwit consensus rules, which go along with it.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T19:11:31.597Z
@PieterWuille I did, but I still do not understand it. It's still too technical for me, I'm trying to understand conceptually. That's why I'm trying this way by asking questions here. After I understand things conceptually I think I will then be able to understand the details.
dassd
CC BY-SA 4.0
2023-09-10T19:13:43.247Z
1) Yes, unknown service flags are ignored by nodes that don't know them. 2) Yes, the marker/flag format is used for any transaction that has at least one non-empty witness (old format is still used for transactions without witnesses) 3) Yes, the witness block format is identical to telhe normal block format, but instead of normal transactions, witness transactions are included.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T19:57:49.453Z
@PieterWuille If old nodes operate with stripped block and new nodes with the, let's call it, "full" blocks, does it mean that old nodes and new nodes will have different blockchain, since their blocks will be different? They will have same blocks in context of  ids (hashes) and transactions, but blocks by their structure will be different?
dassd
CC BY-SA 4.0
2023-09-10T20:07:35.477Z
@joke: It is probably clearer to think of it not as different blockchains but as the same blockchain with Segwit data stripped out. Just as we would say a pruning node and a non-pruning node have the same blockchain even though one node has more data than the other.
RedGrittyBrick
CC BY-SA 4.0
2023-09-10T20:08:07.727Z
Clearly they don't have the exact same data, as the pre-segwit nodes lack the witnesses. But apart from that, whether that's "structurally" different will depend on how you define that.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T20:20:08.553Z
@PieterWuille "structurally" in context that old nodes will have blockchain made of blocks composed of transaction with old serialization format while new nodes will have blockchain made of block composed of transactions with new serialization format (that is, if tansaction has witness input, it will have additionaly marker, flag and witness data, otherwise old serialization format is used too). So blockchains in new and old nodes will differ in this "structurally" way?
dassd
CC BY-SA 4.0
2023-09-10T20:31:10.330Z
@joke This is becoming a semantics discussion, I don't think this will serve anything.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T20:46:09.753Z
@PieterWuille Well its just important for me to know if old nodes have in their blockchain blocks with old serialization tx format, while new nodes have in their blockchain blocks with new serialization format, so thats how their blockchains differ? The other parts of their blockchains are the same.
dassd
CC BY-SA 4.0
2023-09-10T20:53:11.830Z
@joke Yes, exactly. New nodes use the flag/marker serialization (which includes witness data) for transactions that have non-empty witnesses, while old nodes always use the old serialization format. To me however, serialization is just an implementation detail - the question is which data they have, as it can always be converted to another format. And in terms of data, they have exactly the same data, just without witness data.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T21:05:35.170Z
@PieterWuille Yes I understand that. Just I want to understand how will their blockchain differ? Is it as RedGrittyBrick said that in old nodes blockchain will be composed of stripped blocks (old tx serialization format) while new nodes has blockchain composed of blocks with witness data (new tx serialization format)? I am asking because if its like that, then I understand the way that segwit is backward compatible from this perspective.
dassd
CC BY-SA 4.0
2023-09-10T21:07:56.357Z
@Joke I thought I had already confirmed that! Yes, old nodes only use the old serialization, which does not support witnesses, so yes, what they receive are by definition the blocks without the witnesses. They likely also store those blocks in that format, but they don't have to.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T21:48:51.307Z
@PieterWuille Oh, sorry, did not understand it like that. What do you mean by saying 'but they dont have to'? Does it mean that they can store it in some arbitrary way that suit them, I mean how do Bitcoin core nodes do that?
dassd
CC BY-SA 4.0
2023-09-10T22:05:34.030Z
Yes, that's my point, nodes can use whatever format they want to store the data, the only thing that matters is what data they have available and which protocol they speak. Bitcoin Core stores blocks in the same format on disk as it'd sent on the network, packed together with multiple blocks in a files with some metadata before each block, and a separate database that contains an index of all those blocks.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T22:21:49.670Z
@PieterWuille Cool, and only different between old and new nodes in the context of block storage, if I understand correctly (and I hope I finally do), is that old nodes will **only** store blocks with the old ser format, and new nodes will **also include** a block with the new ser format (if there is a segwit transaction in it). **I am talking about bitcoin core.** True?
dassd
CC BY-SA 4.0
2023-09-10T22:25:15.240Z
@joke Bitcoin Core will only store blocks once, using for each transaction the appropriate format (witness format if it has a witness, old format otherwise). If a pre-segwit peer asks for a non-witness block or transaction, the data is re-serialized in the old format (which discards witness data) before sending.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T22:35:57.980Z
@PieterWuille Yes, but that one time when the Bitcoin core node stores the block, if thats **1) old node**: it stores block that consists only of all old serialization format transactions (no matter whether tx is segwit or not; it ifs segwit it will be without witness data/flag/marker and will be considered as anyone-can-spend) **2) new node**: it stores block with old ser tx format and new ser tx format. If and old peer ask a new node for some block or tx, the new node will re-serialized in the old format before sending. Right?
dassd
CC BY-SA 4.0
2023-09-10T22:37:06.160Z
If by "with old ser tx format and new ser tx format" you mean that the blocks it stored will use a mix of both formats, yes. It doesn't store everything twice. Otherwise, indeed!
Pieter Wuille
CC BY-SA 4.0
2023-09-10T22:46:04.773Z
@PieterWuille Oh, I didn't express myself well.  I just wanna say the following. If we have, for example, block with 10 tx where 3 are segwit and 7 are legacy (P2PKH for example), then the old nodes will store the block as 10 tx all serialized with old format (no witness data, no marker, no flag, 3 segwit tx are considered as anyone-can-spend). On the other hand, the new nodes will store the same block as 3 segwit tx with new ser format (yes witness, yes marker, yes flag, these tx are not anyone-can-spend) and for other 7 tx use old tx ser format. Right? This is only difference in storage?
dassd
CC BY-SA 4.0
2023-09-10T22:49:21.217Z
@joke All correct.
Pieter Wuille
CC BY-SA 4.0
2023-09-10T22:59:07.040Z
@PieterWuille Cool, thanks, everything clear.
dassd
CC BY-SA 4.0
2023-09-10T17:56:33.907Z
2023-09-10T17:56:33.907Z
RedGrittyBrick
<p>I understand that SegWit is backward compatible (soft-fork) from <a href="https://bitcoin.stackexchange.com/questions/101142/full-list-of-special-cases-during-bitcoin-script-execution-p2sh-p2wsh-etc/101146?noredirect=1#comment136515_101146">this perspective</a>.</p>
<p>So, for old nodes the following output script (scriptPubKey)</p>
<p><code>OP_n (where n from 0 to 16) &lt;2-40 bytes&gt;</code></p>
<p>... is considered as anyone-can-spend and in &quot;concatenation&quot; with the empty input script (scriptSig) it will be always valid.</p>
<p>For new (updated) nodes it will be considered as SegWit, so the special SegWit validation process, as Pieter Wuille described, will start.</p>
<p>Observing only this, everything is clear to me.</p>
<p>What is not clear to me is how all this will work when the structure of the transaction has been changed from SegWit.</p>
<p>Namely, according to bip144, the structure of the transaction has been changed by &quot;expanding&quot; the transaction with <strong>two bytes</strong> (marker (0x00) and flag (0x01)) after <code>version</code> field, and also adding <strong>witness data</strong> after <code>tx_outs</code> and before <code>locktime</code>.</p>
<p>For new (updated) nodes this is considered as a sign that there is <code>witness data</code>, however, for old nodes this is a transaction with 0 inputs and 1 output which is considered <strong>invalid</strong> so they will always discard this transaction (especially since there is also <code>witness data</code> that is not clear to them at all).</p>
<p>Even if we would consider that the old nodes would not relay these transactions to other peers, they would not even accept them in the blocks they receive from the miners, because for them these are completely invalid transactions (valid from the first perspective of inputs and outpus since they are anyone-can-spend, but invalid from this second, structure perspective).</p>
<p>Can someone explain to me how this works? I misunderstood something.</p>

CC BY-SA 4.0
2023-09-10T18:31:11.007Z
Does this answer your question? [How is SegWit a soft fork?](https://bitcoin.stackexchange.com/questions/52152/how-is-segwit-a-soft-fork)
Pieter Wuille
CC BY-SA 4.0
2023-09-10T18:35:13.713Z
Yes, I read it, but I still don't understand 100%. I have additional questions. I've posted the questions below in a reply, but may as well post them here so RedGrittyBrick doesn't get too many notices.
dassd
<segregated-witness>
How is segwit backward compatible (soft fork) when the transaction serialization structure is changed?
dassd
# How is segwit backward compatible (soft fork) when the transaction serialization structure is changed?

(Asked by: username_0 on 2023-09-10)

I understand that SegWit is backward compatible (soft-fork) from [this perspective](https://bitcoin.stackexchange.com/questions/101142/full-list-of-special-cases-during-bitcoin-script-execution-p2sh-p2wsh-etc/101146?noredirect=1#comment136515_101146).

So, for old nodes the following output script (scriptPubKey)

`OP_n (where n from 0 to 16) <2-40 bytes>`

... is considered as anyone-can-spend and in "concatenation" with the empty input script (scriptSig) it will be always valid.

For new (updated) nodes it will be considered as SegWit, so the special SegWit validation process, as username_1 username_1 described, will start.

Observing only this, everything is clear to me.

What is not clear to me is how all this will work when the structure of the transaction has been changed from SegWit.

Namely, according to bip144, the structure of the transaction has been changed by "expanding" the transaction with **two bytes** (marker (0x00) and flag (0x01)) after `version` field, and also adding **witness data** after `tx_outs` and before `locktime`.

For new (updated) nodes this is considered as a sign that there is `witness data`, however, for old nodes this is a transaction with 0 inputs and 1 output which is considered **invalid** so they will always discard this transaction (especially since there is also `witness data` that is not clear to them at all).

Even if we would consider that the old nodes would not relay these transactions to other peers, they would not even accept them in the blocks they receive from the miners, because for them these are completely invalid transactions (valid from the first perspective of inputs and outpus since they are anyone-can-spend, but invalid from this second, structure perspective).

Can someone explain to me how this works? I misunderstood something.

**Tags:** segregated-witness

> **Comments:**
> **username_1**: Does this answer your question? [How is SegWit a soft fork?](https://bitcoin.stackexchange.com/questions/52152/how-is-segwit-a-soft-fork)  
> **username_0**: Yes, I read it, but I still don't understand 100%. I have additional questions. I've posted the questions below in a reply, but may as well post them here so username_2 doesn't get too many notices.  

---

## 1 answers

### Answer by username_2 on 2023-09-10. Score: 3 | Accepted answer

Old nodes never see the new serialisation format

When an old node asks a new node for data, the new node edits the data into the old serialisation format before sending it

> **Comments:**
> **username_0**: Okay, but when a node just want to relay the transaction it got to all its peers, how will he know if the peer is old or new (updated) node?  
> **username_2**: Part of the communications handshake distinguishes segwit nodes  
> **username_0**: Okay, I understand they can do that when we're talking about relaying. But what about the block transaction received by the miner? How will the old nodes confirm this since they don't understand the new structure? I mean, it can't be the old serialization format without witnesses and markers/flags because then the old and new nodes would have a different blockchain and this block would have a different hash?  
> **username_1**: @username_2 While that's true, the handshake just lets the other side know that the peer can *offer* witness. To actually get witness data, a node has to ask for a witness_tx or a witness_block, rather than tx or block (if you ask for those, even from a segwit node, you get the stripped version).  
> **username_1**: @joke The txid of a transaction is independent of whether the witnesses are included or not, and the hash of a block is the hash of its header - which also doesn't include the witnesses. Witnesses data *is* committed to in blocks, but just by including the hash of all the witnesses in the coinbase transaction somewhere. So no, you can just drop all the witness data from a witness block, give it to a pre-segwit peer, and it will be valid to them.  
> **username_0**: @username_1 So the stripped version (of both the block and the transaction) is always sent, with the fact that during handshaking, the segWit node somehow announces itself (it will set some flag or something) as segWit node and that can also send witness data?  
> **username_1**: Right. NODE_WITNESS announces that witnesses are available, and if so, the `getdata` command can be used to ask for transactions, blocks, or for transactions+witness, or for block+witness. Without that flag, only transactions or blocks can be asked for.  
> **username_0**: @username_1 Couple of question. 1) I assume that NODE_WITNESS flag is introduced with segWit, so it won't "confuse" old nodes? I mean, they will just ignore it and ask for `tx`/`block` (not `tx+witness`/`block+witness`)? 2) If we ask for `tx+witness`, we will get from node new serialization format (with marker+flag+witness), and if we ask for `tx` we get old serialization format? (of course all this in case that tx is segWit) 3) What does it mean `block+witness` and `block`? Is first that we get block consist of all tx with new ser format, while the second one is all tx with old ser format?  
> **username_1**: @joke To go in this much depth, I suggest you just read BIP144 which specifies the protocol/serialization changes. BIP141 specifies the segwit consensus rules, which go along with it.  
> **username_0**: @username_1 I did, but I still do not understand it. It's still too technical for me, I'm trying to understand conceptually. That's why I'm trying this way by asking questions here. After I understand things conceptually I think I will then be able to understand the details.  
> **username_1**: 1) Yes, unknown service flags are ignored by nodes that don't know them. 2) Yes, the marker/flag format is used for any transaction that has at least one non-empty witness (old format is still used for transactions without witnesses) 3) Yes, the witness block format is identical to telhe normal block format, but instead of normal transactions, witness transactions are included.  
> **username_0**: @username_1 If old nodes operate with stripped block and new nodes with the, let's call it, "full" blocks, does it mean that old nodes and new nodes will have different blockchain, since their blocks will be different? They will have same blocks in context of  ids (hashes) and transactions, but blocks by their structure will be different?  
> **username_2**: @joke: It is probably clearer to think of it not as different blockchains but as the same blockchain with Segwit data stripped out. Just as we would say a pruning node and a non-pruning node have the same blockchain even though one node has more data than the other.  
> **username_1**: Clearly they don't have the exact same data, as the pre-segwit nodes lack the witnesses. But apart from that, whether that's "structurally" different will depend on how you define that.  
> **username_0**: @username_1 "structurally" in context that old nodes will have blockchain made of blocks composed of transaction with old serialization format while new nodes will have blockchain made of block composed of transactions with new serialization format (that is, if tansaction has witness input, it will have additionaly marker, flag and witness data, otherwise old serialization format is used too). So blockchains in new and old nodes will differ in this "structurally" way?  
> **username_1**: @joke This is becoming a semantics discussion, I don't think this will serve anything.  
> **username_0**: @username_1 Well its just important for me to know if old nodes have in their blockchain blocks with old serialization tx format, while new nodes have in their blockchain blocks with new serialization format, so thats how their blockchains differ? The other parts of their blockchains are the same.  
> **username_1**: @joke Yes, exactly. New nodes use the flag/marker serialization (which includes witness data) for transactions that have non-empty witnesses, while old nodes always use the old serialization format. To me however, serialization is just an implementation detail - the question is which data they have, as it can always be converted to another format. And in terms of data, they have exactly the same data, just without witness data.  
> **username_0**: @username_1 Yes I understand that. Just I want to understand how will their blockchain differ? Is it as username_2 said that in old nodes blockchain will be composed of stripped blocks (old tx serialization format) while new nodes has blockchain composed of blocks with witness data (new tx serialization format)? I am asking because if its like that, then I understand the way that segwit is backward compatible from this perspective.  
> **username_1**: @Joke I thought I had already confirmed that! Yes, old nodes only use the old serialization, which does not support witnesses, so yes, what they receive are by definition the blocks without the witnesses. They likely also store those blocks in that format, but they don't have to.  
> **username_0**: @username_1 Oh, sorry, did not understand it like that. What do you mean by saying 'but they dont have to'? Does it mean that they can store it in some arbitrary way that suit them, I mean how do Bitcoin core nodes do that?  
> **username_1**: Yes, that's my point, nodes can use whatever format they want to store the data, the only thing that matters is what data they have available and which protocol they speak. Bitcoin Core stores blocks in the same format on disk as it'd sent on the network, packed together with multiple blocks in a files with some metadata before each block, and a separate database that contains an index of all those blocks.  
> **username_0**: @username_1 Cool, and only different between old and new nodes in the context of block storage, if I understand correctly (and I hope I finally do), is that old nodes will **only** store blocks with the old ser format, and new nodes will **also include** a block with the new ser format (if there is a segwit transaction in it). **I am talking about bitcoin core.** True?  
> **username_1**: @joke Bitcoin Core will only store blocks once, using for each transaction the appropriate format (witness format if it has a witness, old format otherwise). If a pre-segwit peer asks for a non-witness block or transaction, the data is re-serialized in the old format (which discards witness data) before sending.  
> **username_0**: @username_1 Yes, but that one time when the Bitcoin core node stores the block, if thats **1) old node**: it stores block that consists only of all old serialization format transactions (no matter whether tx is segwit or not; it ifs segwit it will be without witness data/flag/marker and will be considered as anyone-can-spend) **2) new node**: it stores block with old ser tx format and new ser tx format. If and old peer ask a new node for some block or tx, the new node will re-serialized in the old format before sending. Right?  
> **username_1**: If by "with old ser tx format and new ser tx format" you mean that the blocks it stored will use a mix of both formats, yes. It doesn't store everything twice. Otherwise, indeed!  
> **username_0**: @username_1 Oh, I didn't express myself well.  I just wanna say the following. If we have, for example, block with 10 tx where 3 are segwit and 7 are legacy (P2PKH for example), then the old nodes will store the block as 10 tx all serialized with old format (no witness data, no marker, no flag, 3 segwit tx are considered as anyone-can-spend). On the other hand, the new nodes will store the same block as 3 segwit tx with new ser format (yes witness, yes marker, yes flag, these tx are not anyone-can-spend) and for other 7 tx use old tx ser format. Right? This is only difference in storage?  
> **username_1**: @joke All correct.  
> **username_0**: @username_1 Cool, thanks, everything clear.  

---

