<p>Bitcoin is designed with the idea of annonymity of nodes in mind.  This means that nodes do <em>not</em> have identities on the network.  They don't even have pseudononymous identities in the same way that addresses are pseudononymous.  This identity-less system is central how Bitcoin works.  By introducing the concept of identity, a whole new class of problems are introduced...namely authentication and trust.</p>

<p>Let's pretend that full nodes are keeping an index of which nodes have full copies of which blocks.  Because this index is created from the messages sent from <strong>unauthenticated</strong> peers, it leaves this index vulnerable to cache poisoning attacks.  There is no reason to trust that a certain node actually contains (or retains) the transactions in the block it claims it has.  There is also no reason to believe that a certain node <em>is</em> the node it claims to be.  If all full nodes on the system were relying on this unauthenticated trust, an attacker could essentially delete the block details (transactions) from the entire network by tricking all the nodes into thinking some other node has the data.</p>

<p>Luckily, this is not how Bitcoin works, and full nodes do not need to trust any specific peer.  Instead, full nodes independently validate all transactions in all blocks, thus validating the entire chain.  When a block is validated, and the transactions in that block have no UTXO, the node can optionally prune the data.  Nodes currently rely on other nodes to fill in the details of historical blocks, but the key difference is that they don't rely on <em>specific</em> nodes in order to do so.  Being able to identify and query a specific node is not required, nor desired.</p>

CC BY-SA 3.0
2016-07-25T16:01:44.263Z
Sorry. What I meant by "identity" is the node address as specified in the protocol documentation. The message advertising the presence of a block on a node is a standard inv message. I believe that answers the issue. I edit the proposal accordingly.
Gérard Dréan
CC BY-SA 3.0
2016-07-25T16:09:36.953Z
That doesn't fix things.  It doesn't matter how a node identifies other nodes, whether by IP address or a unique ID field.  The problem is that nodes now rely on a *specific* node, regardless of how it is indentified.  The problem is that you'd be querying a specific node instead of the network as a whole.  This is what introduces all the authentication and trust issues.  The protocol has to work in such a way that it doesn't matter if a specific node is lying to you or goes offline.  I don't think you've solved that.
Jestin
CC BY-SA 3.0
2016-07-25T19:12:34.223Z
I am puzzled. I assume that if a node sends me an inv message saying "I have object X" I may now ask ** that specific node** to send me that object. Am I wrong ? if not, what is the difference with my proposal ?
Gérard Dréan
CC BY-SA 3.0
2016-07-25T19:42:40.143Z
Perhaps I'm misunderstanding your idea, but it sounds like you'd be relying on a specific node for a specific piece of data.  You can ask, but what if that node is lying?  What if that node goes offline?  Asking is just fine, but you shouldn't *trust* that the answer will be there, nor that it is accurate.  A full node that relies on that level of trust with another node, is not really a full node.  I think @Pieter's answer summed this up fairly well.
Jestin
CC BY-SA 3.0
2016-07-25T21:31:17.733Z
Trust is the problem. Bitcoin was founded based on the idea that you should trust no one literally. Your proposal gives room to a potential collusion attack where the subset of nodes you're connected to may potentially feed your node with invalid information as your proposal involves the delegation of trust to some "super nodes" which store the full payload. But then, if we can have some trust then what is the point of a blockchain again?
rny
CC BY-SA 3.0
2016-07-26T08:19:24.257Z
@Jestin " Being able to identify and query a specific node is not required, nor desired"  Are you saying that getdata always queries the entire network ?
Gérard Dréan
CC BY-SA 3.0
2016-07-26T11:48:05.763Z
No, just that trusting single nodes is not how Bitcoin works.
Jestin
CC BY-SA 3.0
2016-07-26T12:11:48.933Z
And that is NOT how my suggestion would work either. All I am saying is that there are many occasions where a node identifies and queries  another node in the current system. How it uses the response is another topic.
Gérard Dréan
CC BY-SA 3.0
2016-07-26T12:14:30.120Z
but please see my answers to Pieter Wuille about 2 hours ago.
Gérard Dréan
CC BY-SA 3.0
2016-07-26T12:20:09.020Z
After re-reading your question and comments, I have to ask: do you think this forum is where Bitcoin improvement proposals are made?  I apologize if I'm assuming too much.
Jestin
CC BY-SA 3.0
2016-07-27T12:37:13.070Z
Well, frankly no. But I am not sure that my idea deserves the formality of a BIP, or event that is is valid, all things considered. That's what I am trying to find out.
Gérard Dréan
CC BY-SA 3.0
2016-07-27T12:48:31.330Z
Sorry, I had to check.  We get a lot of people mistaking this forum for things it is not.
Jestin
CC BY-SA 3.0
2016-07-25T15:25:07.953Z
2016-07-25T19:47:28.440Z
2016-07-25T19:47:28.440Z
Jestin
<p>One thing you're missing is that nodes don't actually need access to blocks.</p>

<p>The only time a "scan of the blockchain" occurs is to give historical blocks to another node that is syncing, or to find historical transactions a wallet missed. Solutions for those case are not hard. Sequential bulk storage and bandwidth are cheap and it's not something every node on the internet needs to provide to everyone. You can find movies on the internet for free that take not much less space than the block chain. </p>

<p>Modern full nodes can even run without keeping blocks on disk at all (see the <code>-prune</code> option in Bitcoin Core). In this mode, all blocks are still downloaded and verified one by one, but not kept on disk after verification. The only thing that is kept is the UTXO set - the database of the current state of the ledger, rather than its entire history. This is sufficient for validating future transactions and blocks, participating fully in the P2P network, and provide wallet functionality.</p>

<p>So can't we use your idea for the UTXO set instead then? Maybe, but it's pretty pointless. The reason you run a full node in the first place is to be able to verify that nobody is violating any of the rules. If you're willing to not do that completely anymore and instead start relying on trusting some of your peers, why bother? Bitcoin provides a lightweight mode where you don't need access to either the blockchain or the UTXO set at all.</p>

CC BY-SA 3.0
2016-07-25T19:18:58.347Z
Agreed, but **some** nodes must keep the entire blockchain. I understand that presently many thousands do, and that their number is considered problematically low by some.Those are the nodes I am talking about. Now you say that the size of the blockchain is not a problem for those. II have every respect for your opinion, but I have also read comments to the contrary.
Gérard Dréan
CC BY-SA 3.0
2016-07-26T09:37:48.010Z
In a nutshell, my suggestion would introduce an intermediate status between full node and thin client,"half-full" nodes being able to keep summaries of certain blocks.
The local blockchain would include a mix of full blocks and summary . Peers would be called upon when the full contents of a summary block are needed. That would be best implemented by using a derivative of the SPV protocol on a block-by-block basis rather than for the entire blockchain. There should be a mechanism to ensure that a node always keeps certain blocks in full format (that might turn out to be a tricky part).
Gérard Dréan
CC BY-SA 3.0
2016-07-26T09:38:24.857Z
My hope is that, even if the number of full nodes decreases, the number of « half-full » nodes will more than compensate, and that for each block in the blockchain, the number of nodes with a reliable copy of that block will increase.

That being said, I am quite prepared to accept that, after going through the likely behaviour of node operators and through the arithmetic of additional storage required vs storage saved, plus additional processing required, the whole proposition may be pointless as you suggest. I feel personally unable to go through that reasoning, for lack of relevant data.
Gérard Dréan
CC BY-SA 3.0
2016-07-25T18:34:37.917Z
2016-07-25T18:34:37.917Z
Pieter Wuille
<p>As far as I know that could be done but we would still needing full nodes to prevent missing blocks.</p>

<p>Just as there are software clients that run with no copy of the block chain there could be clients with a partial copy.</p>

CC BY-SA 3.0
2016-07-24T17:07:39.433Z
Absolutely. A provision for ensuring that there is a sufficient number of full copies of every block would be required, but I assume that is not difficult to implement.
Gérard Dréan
CC BY-SA 3.0
2016-07-24T16:56:55.580Z
2016-07-24T16:56:55.580Z
Josu Goñi
<p>The following idea is so simple that I cannot believe that it is really new. It must have been presented and discussed at length (and rejected) previously, but I submit it nevertheless, for lack of having found any mention of such discussions.</p>

<p>The basic idea is simple : when a (full) node successfully adds a block to its copy of the blockchain, it broadcasts an inv message giving its address and the identity of the block.</p>

<p>Every node is thus able to maintain for every block a list of nodes having that same block in their blockchain. That inventory will normally be separate from the local blockchain.</p>

<p>Now every node can decide, based on local considerations, whether to include a block in its local blockchain or to only include a small placeholder and call upon other nodes if and when the full contents of that block are required. That would be a base for a genuinely “distributed” blockchain.</p>

<p>In practice, all nodes would probably want to initially include all full blocks, since at that time they would not have identified enough nodes carrying the same block. A capability to replace an existing block with a reference to other nodes would therefore be required. To that effect, block headers should include a reference to the (local) list of alternate source nodes for that block.</p>

<p>A node could then simply erase the payload of a node in its copy of the blockchain, retaining only the header unchanged, including the hash of the full block and of the previous block to retain continuity of the chain. Of course that should be broadcast so that the ofher nodes update their local table of alternate sources.</p>

<p>When a scan of the blockchain hits a block with an empty payload, the node could return a "block unavailable" answer (for instance if multiple nodes are being queried), or the local list of alternate sources for that block could be used to retrieve a copy (or many copies for verification purposes), which could be checked using the hash retained in the abbreviated header.</p>

<p>Some mechanism for ensuring that each block exists in full version on a sufficient number of nodes would be required. </p>

<p>What is wrong with that proposal ?</p>

<blockchain>
A proposal for a genuinely distributed blockchain and pruning
Gérard Dréan
# A proposal for a genuinely distributed blockchain and pruning

(Asked by: username_0 on 2016-07-24)

The following idea is so simple that I cannot believe that it is really new. It must have been presented and discussed at length (and rejected) previously, but I submit it nevertheless, for lack of having found any mention of such discussions.

The basic idea is simple : when a (full) node successfully adds a block to its copy of the blockchain, it broadcasts an inv message giving its address and the identity of the block.

Every node is thus able to maintain for every block a list of nodes having that same block in their blockchain. That inventory will normally be separate from the local blockchain.

Now every node can decide, based on local considerations, whether to include a block in its local blockchain or to only include a small placeholder and call upon other nodes if and when the full contents of that block are required. That would be a base for a genuinely “distributed” blockchain.

In practice, all nodes would probably want to initially include all full blocks, since at that time they would not have identified enough nodes carrying the same block. A capability to replace an existing block with a reference to other nodes would therefore be required. To that effect, block headers should include a reference to the (local) list of alternate source nodes for that block.

A node could then simply erase the payload of a node in its copy of the blockchain, retaining only the header unchanged, including the hash of the full block and of the previous block to retain continuity of the chain. Of course that should be broadcast so that the ofher nodes update their local table of alternate sources.

When a scan of the blockchain hits a block with an empty payload, the node could return a "block unavailable" answer (for instance if multiple nodes are being queried), or the local list of alternate sources for that block could be used to retrieve a copy (or many copies for verification purposes), which could be checked using the hash retained in the abbreviated header.

Some mechanism for ensuring that each block exists in full version on a sufficient number of nodes would be required.

What is wrong with that proposal ?

**Tags:** blockchain


---

## 3 answers

### Answer by username_1 on 2016-07-25. Score: 2

Bitcoin is designed with the idea of annonymity of nodes in mind. This means that nodes do *not* have identities on the network. They don't even have pseudononymous identities in the same way that addresses are pseudononymous. This identity-less system is central how Bitcoin works. By introducing the concept of identity, a whole new class of problems are introduced...namely authentication and trust.

Let's pretend that full nodes are keeping an index of which nodes have full copies of which blocks. Because this index is created from the messages sent from **unauthenticated** peers, it leaves this index vulnerable to cache poisoning attacks. There is no reason to trust that a certain node actually contains (or retains) the transactions in the block it claims it has. There is also no reason to believe that a certain node *is* the node it claims to be. If all full nodes on the system were relying on this unauthenticated trust, an attacker could essentially delete the block details (transactions) from the entire network by tricking all the nodes into thinking some other node has the data.

Luckily, this is not how Bitcoin works, and full nodes do not need to trust any specific peer. Instead, full nodes independently validate all transactions in all blocks, thus validating the entire chain. When a block is validated, and the transactions in that block have no UTXO, the node can optionally prune the data. Nodes currently rely on other nodes to fill in the details of historical blocks, but the key difference is that they don't rely on *specific* nodes in order to do so. Being able to identify and query a specific node is not required, nor desired.

> **Comments:**
> **username_0**: Sorry. What I meant by "identity" is the node address as specified in the protocol documentation. The message advertising the presence of a block on a node is a standard inv message. I believe that answers the issue. I edit the proposal accordingly.  
> **username_1**: That doesn't fix things.  It doesn't matter how a node identifies other nodes, whether by IP address or a unique ID field.  The problem is that nodes now rely on a *specific* node, regardless of how it is indentified.  The problem is that you'd be querying a specific node instead of the network as a whole.  This is what introduces all the authentication and trust issues.  The protocol has to work in such a way that it doesn't matter if a specific node is lying to you or goes offline.  I don't think you've solved that.  
> **username_0**: I am puzzled. I assume that if a node sends me an inv message saying "I have object X" I may now ask ** that specific node** to send me that object. Am I wrong ? if not, what is the difference with my proposal ?  
> **username_1**: Perhaps I'm misunderstanding your idea, but it sounds like you'd be relying on a specific node for a specific piece of data.  You can ask, but what if that node is lying?  What if that node goes offline?  Asking is just fine, but you shouldn't *trust* that the answer will be there, nor that it is accurate.  A full node that relies on that level of trust with another node, is not really a full node.  I think @username_3's answer summed this up fairly well.  
> **username_2**: Trust is the problem. Bitcoin was founded based on the idea that you should trust no one literally. Your proposal gives room to a potential collusion attack where the subset of nodes you're connected to may potentially feed your node with invalid information as your proposal involves the delegation of trust to some "super nodes" which store the full payload. But then, if we can have some trust then what is the point of a blockchain again?  
> **username_0**: @username_1 " Being able to identify and query a specific node is not required, nor desired"  Are you saying that getdata always queries the entire network ?  
> **username_1**: No, just that trusting single nodes is not how Bitcoin works.  
> **username_0**: And that is NOT how my suggestion would work either. All I am saying is that there are many occasions where a node identifies and queries  another node in the current system. How it uses the response is another topic.  
> **username_0**: but please see my answers to username_3 username_3 about 2 hours ago.  
> **username_1**: After re-reading your question and comments, I have to ask: do you think this forum is where Bitcoin improvement proposals are made?  I apologize if I'm assuming too much.  
> **username_0**: Well, frankly no. But I am not sure that my idea deserves the formality of a BIP, or event that is is valid, all things considered. That's what I am trying to find out.  
> **username_1**: Sorry, I had to check.  We get a lot of people mistaking this forum for things it is not.  

---

### Answer by username_3 on 2016-07-25. Score: 2

One thing you're missing is that nodes don't actually need access to blocks.

The only time a "scan of the blockchain" occurs is to give historical blocks to another node that is syncing, or to find historical transactions a wallet missed. Solutions for those case are not hard. Sequential bulk storage and bandwidth are cheap and it's not something every node on the internet needs to provide to everyone. You can find movies on the internet for free that take not much less space than the block chain.

Modern full nodes can even run without keeping blocks on disk at all (see the `-prune` option in Bitcoin Core). In this mode, all blocks are still downloaded and verified one by one, but not kept on disk after verification. The only thing that is kept is the UTXO set - the database of the current state of the ledger, rather than its entire history. This is sufficient for validating future transactions and blocks, participating fully in the P2P network, and provide wallet functionality.

So can't we use your idea for the UTXO set instead then? Maybe, but it's pretty pointless. The reason you run a full node in the first place is to be able to verify that nobody is violating any of the rules. If you're willing to not do that completely anymore and instead start relying on trusting some of your peers, why bother? Bitcoin provides a lightweight mode where you don't need access to either the blockchain or the UTXO set at all.

> **Comments:**
> **username_0**: Agreed, but **some** nodes must keep the entire blockchain. I understand that presently many thousands do, and that their number is considered problematically low by some.Those are the nodes I am talking about. Now you say that the size of the blockchain is not a problem for those. II have every respect for your opinion, but I have also read comments to the contrary.  
> **username_0**: In a nutshell, my suggestion would introduce an intermediate status between full node and thin client,"half-full" nodes being able to keep summaries of certain blocks. The local blockchain would include a mix of full blocks and summary . Peers would be called upon when the full contents of a summary block are needed. That would be best implemented by using a derivative of the SPV protocol on a block-by-block basis rather than for the entire blockchain. There should be a mechanism to ensure that a node always keeps certain blocks in full format (that might turn out to be a tricky part).  
> **username_0**: My hope is that, even if the number of full nodes decreases, the number of « half-full » nodes will more than compensate, and that for each block in the blockchain, the number of nodes with a reliable copy of that block will increase.  That being said, I am quite prepared to accept that, after going through the likely behaviour of node operators and through the arithmetic of additional storage required vs storage saved, plus additional processing required, the whole proposition may be pointless as you suggest. I feel personally unable to go through that reasoning, for lack of relevant data.  

---

### Answer by username_4 on 2016-07-24. Score: 0

As far as I know that could be done but we would still needing full nodes to prevent missing blocks.

Just as there are software clients that run with no copy of the block chain there could be clients with a partial copy.

> **Comments:**
> **username_0**: Absolutely. A provision for ensuring that there is a sufficient number of full copies of every block would be required, but I assume that is not difficult to implement.  

---

