<p>I think all big miners have own developed software to manage mining process and able to switch to recently received block header before validation is complete. Block validation process should not take a lot of time, because most of transactions from block already in mempool and already verified. Validation task is verify missed transactions, verify coinbase, verify block reward, verify merkleroot. So block validation should complete in few seconds. 
Block to be propagated through the network, depends on connection speed.</p>

CC BY-SA 3.0
2016-12-28T13:25:33.590Z
2016-12-28T13:25:33.590Z
bitaps.com
<p>As stated above many miners today already start mining using only the new blockheader before the block is validated and downloaded. </p>

<p>This is why you sometimes see newly generated block with only one transaction (the coinbase transaction). An example of this is the following block:
<a href="https://blockchain.info/block/00000000000000000a06dbd18a15a452c4dd50f662044e654f83066da2775ed8" rel="nofollow noreferrer">https://blockchain.info/block/00000000000000000a06dbd18a15a452c4dd50f662044e654f83066da2775ed8</a></p>

<p>This is because the miner does not know exactly which transactions were included in the previous block before downloading and validating it. Because of this, it only includes the coinbase transaction until the mempool has been updated to avoid including a transaction that has already been included in the last block. </p>

CC BY-SA 3.0
2016-12-28T13:36:36.750Z
2016-12-28T13:36:36.750Z
user44404
<p>There are several questions here.</p>
<blockquote>
<p>Please correct me if I'm wrong: The miner validates the newly received block before using it themself and sending it to their other connected peers.</p>
</blockquote>
<p>Yes and no. Note that by miner we're talking about people who build blocks themselves - that includes solo miners, pool operators, and p2pool users. Hashers that only connect to a pool and perform work are not part of that group.</p>
<p>Miners can - and sometimes do - build a new block before they've fully processed the previous one (even if it's their own), to avoid time not mining. Unfortunately, as they don't yet know which UTXOs are spent in the just-received block, they don't know what follow-up transactions are valid, and thus can't include any new transactions. Because of this, these blocks will be empty apart from the coinbase.</p>
<p>In practice, these miners will have two mechanisms to update the proposed block their hashers are grinding on:</p>
<ol>
<li>A new best block hash is announced within their own network (or detected on another pool's network - for example by listening to their pool interface, or by an agreemwnt to send each other information). When this happens, all hashers are told to start working on an empty block.</li>
<li>A new block is received through their <code>bitcoind</code> (via the P2P protocol, via the <code>submitblock</code> RPC command for their blocks, or via separate relay networks like <a href="http://bitcoinfibre.org" rel="nofollow noreferrer">FIBRE</a>). <code>bitcoind</code> then builds a new set of valid transaction on top (through the <code>getblocktemplate</code> RPC), and they update their hashers to start working on a block with these transactions.</li>
</ol>
<p>The assumption is that when (1) happens, the same block will shortly go through (2), and we'll switch from working on an empty block on top to a normal block on top. There was once an incident where this did not happen.</p>
<p>When BIP66 activated, some miners running a BIP66-enabled <code>bitcoind</code> versions were listening in on blocks sent by pre-BIP66 pools. A pre-BIP66 miner produced a non-BIP66-compliant block (wrong version numbers), and BIP66-enabled miners listened in, and started producing empty blocks on top. Of course, their <code>bitcoind</code> never accepted the full block as it was invalid according to the new rules - rules those themselves miners agreed to. The result was a sequence of many empty blocks on top, with many miners building on top of the previous invalid blocks, all of which were not accepted by the network.</p>
<p>So to answer your question:</p>
<blockquote>
<p>So why keep mining on top of the previous block?</p>
</blockquote>
<p>Because the new one may be invalid. It's unlikely to happen intentionally due to the costs of mining an invalid block, but it can happen as a result of software or manual bugs. Furthermore, we should not build infrastructure that relies on this not happening - as doing so might over time make such attacks cheaper.</p>
<blockquote>
<p>How long does this process typically take? What does it depend on?</p>
</blockquote>
<p>In your process you're only counting block validation. But the whole process consists of everything between (A) the creation of a valid block on the network and (B) hashers switching to building a block on top of it. This includes:</p>
<ul>
<li><p>The previous block creator <em>getting the block out to the network</em>. There may be unintentional delays here, or even intentional delays (like a Selfish Mining attack).</p>
</li>
<li><p>The blocks needs to <em>propagate across the network</em>. Normal <code>bitcoind</code> nodes only propagate after full validation, and require bursts of high bandwidth to transfer all blocks. More recent technology like Compact Blocks (BIP152) and FIBRE avoid full resubmission or even waiting until validation completed.</p>
</li>
<li><p>The blocks need to be <em>validated</em>.</p>
</li>
<li><p>A new set of valid <em>transactions on top</em> has to be created.</p>
</li>
</ul>
<p>Validation depends on many factors:</p>
<ul>
<li><em>Software version</em>. There are constantly improvements to validation speed.</li>
<li><em>UTXO cache size</em>. The larger the cache, the less database access is needed to retrieve information about the outputs being spent. As a result, just fetching of inputs can take from a few milliseconds to a few seconds.</li>
<li><em>Signature cache size and CPU speed</em>. The larger the cache, the more signature validations can be avoided. These validations - depending on the software version and hardware can vary from 0.01ms to 0.6ms per signature (45ms to 2.7s for a block).</li>
<li><em>Correlation between the memory pool and the new block</em>. If a block contains transactions that a node hasn't seen before, its inputs and signatures are less likely to be cached before.</li>
<li><em>Bandwidth</em>. Prior to Bitcoin Core 0.13, blocks were always submitted in full between peers, which can cause big spikes at the time a new block is announced.</li>
<li><em>Network latency</em>. In more isolated parts of the world, even with good bandwidth, the time it takes for a network packet to reach the outside world can be significant. Depending on the protocol, 1 to 3 roundtrips are needed to send a block. If the latency between two peers is 200ms, that can already mean 1.2s wasted on going back and forth.</li>
<li><em>Number of connections</em>. If a node has many connections, it will try to broadcast a new block simultaneously to all, causing a spike of work of network activity. This can be too much for the CPU or network hardware or bandwidth to handle, causing slow downs when many connections exists.</li>
</ul>
<p>The time to construct a new block mostly depends on software version. In older versions it was up to a few seconds, but lately is has been reduced to tens of milliseconds.</p>
<blockquote>
<p>Is it done on specialized mining hardware or on a general-purpose CPU?</p>
</blockquote>
<p>As far as I know, no custom hardware exists for block validation or block construction.</p>
<blockquote>
<p>Is it correct that the time block validation takes is linear to the block's + witness's size should segwit activate?</p>
</blockquote>
<p>Mostly. There exists an inefficiency in the currently signature hashing algorithm that can be O(n^2) in the size of transactions. This can result in single transactions that take minutes to just compute the signature hash. This is fixed in BIP144, which is always used inside SegWit transaction inputs, making it O(n) at worst (less than 10ms for a block in the worst case on common hardware).</p>
<p>Longer term, there are other factors that play, like the size of the UTXO set. If this would grow to multiple gigabytes, and not fit in typical memory caches, the UTXO fetch time for validation could go up dramatically.</p>
<blockquote>
<p>Also: How long does it take on average for a block to be propagated through the network? It'd be great if that average is weighted on the receiving miners' hashing powers.</p>
</blockquote>
<p>That's complicated. It's certainly not proportional to hashing speed, but more related to network topology and used technology. The FIBRE website has some statistics, but often transfers in less than 20ms more than the minimum theoretical network latency (speed of light across long connections) across the globe. This is only possible by being a private network that assumes its participants will not engage in DoS attacks on the network. The public network is much more robust, but often takes many seconds to propagate a block to large portions of nodes, and dozens of seconds to reach less connected nodes.</p>

CC BY-SA 3.0
2016-12-31T01:00:28.323Z
Do you know whether some kind of slow loris attack is possible? That'd be the miner of a block sending a block header to their peers so they know there is a new block but then taking really long to send the rest of the block. I expect there to be maximum time span until a block is fully received. Do you know it? Furthermore, this begs the quesiton of whether a miner who found a block without fully having received the previous one is able to send their new block to their peers. Will they accept the newly mined block even if they don't have its parent either?
UTF-8
CC BY-SA 3.0
2016-12-31T14:41:00.533Z
That's probably possible (I don't know the exact details of these validationless mining setups), and it would be a good reason not to do validationless mining.
Pieter Wuille
CC BY-SA 4.0
2016-12-28T15:22:30.397Z
2024-12-12T18:13:11.627Z
2024-12-12T18:13:11.627Z
Pieter Wuille
<p>Assume a miner receives a new block from a connected peer. Please correct me if I'm wrong: The miner validates the newly received block before using it themself and sending it to their other connected peers. I heard this but it seems to me that it'd be a better strategy to already use the block to mine on top of it because the block header and the proof of work associated with it is very fast to verify and it's highly unlikely someone made a block header with a small hash for an invalid block. So why keep mining on top of the previous block?</p>

<p>How long does this process typically take? What does it depend on? Is it done on specialized mining hardware or on a general-purpose CPU?</p>

<p>Is it correct that the time block validation takes is linear to the block's + witness's size should segwit activate?</p>

<p>Also: How long does it take on average for a block to be propagated through the network? It'd be great if that average is weighted on the receiving miners' hashing powers.</p>

CC BY-SA 3.0
2016-12-28T13:31:18.283Z
Mining pool can start mining on the top of new block when recieves "inv" block packet from one of peers. It saves time for downloading 1 MiB data.
amaclin
<blockchain><full-node><validation><block-validity>
How long does block validation take?
UTF-8
# How long does block validation take?

(Asked by: username_0 on 2016-12-23)

Assume a miner receives a new block from a connected peer. Please correct me if I'm wrong: The miner validates the newly received block before using it themself and sending it to their other connected peers. I heard this but it seems to me that it'd be a better strategy to already use the block to mine on top of it because the block header and the proof of work associated with it is very fast to verify and it's highly unlikely someone made a block header with a small hash for an invalid block. So why keep mining on top of the previous block?

How long does this process typically take? What does it depend on? Is it done on specialized mining hardware or on a general-purpose CPU?

Is it correct that the time block validation takes is linear to the block's + witness's size should segwit activate?

Also: How long does it take on average for a block to be propagated through the network? It'd be great if that average is weighted on the receiving miners' hashing powers.

**Tags:** blockchain, full-node, validation, block-validity

> **Comments:**
> **username_1**: Mining pool can start mining on the top of new block when recieves "inv" block packet from one of peers. It saves time for downloading 1 MiB data.  

---

## 3 answers

### Answer by username_4 on 2016-12-28. Score: 14 | Accepted answer

There are several questions here.

> Please correct me if I'm wrong: The miner validates the newly received block before using it themself and sending it to their other connected peers.

Yes and no. Note that by miner we're talking about people who build blocks themselves - that includes solo miners, pool operators, and p2pool users. Hashers that only connect to a pool and perform work are not part of that group.

Miners can - and sometimes do - build a new block before they've fully processed the previous one (even if it's their own), to avoid time not mining. Unfortunately, as they don't yet know which UTXOs are spent in the just-received block, they don't know what follow-up transactions are valid, and thus can't include any new transactions. Because of this, these blocks will be empty apart from the coinbase.

In practice, these miners will have two mechanisms to update the proposed block their hashers are grinding on:

1. A new best block hash is announced within their own network (or detected on another pool's network - for example by listening to their pool interface, or by an agreemwnt to send each other information). When this happens, all hashers are told to start working on an empty block.
2. A new block is received through their `bitcoind` (via the P2P protocol, via the `submitblock` RPC command for their blocks, or via separate relay networks like [FIBRE](http://bitcoinfibre.org)). `bitcoind` then builds a new set of valid transaction on top (through the `getblocktemplate` RPC), and they update their hashers to start working on a block with these transactions.

The assumption is that when (1) happens, the same block will shortly go through (2), and we'll switch from working on an empty block on top to a normal block on top. There was once an incident where this did not happen.

When BIP66 activated, some miners running a BIP66-enabled `bitcoind` versions were listening in on blocks sent by pre-BIP66 pools. A pre-BIP66 miner produced a non-BIP66-compliant block (wrong version numbers), and BIP66-enabled miners listened in, and started producing empty blocks on top. Of course, their `bitcoind` never accepted the full block as it was invalid according to the new rules - rules those themselves miners agreed to. The result was a sequence of many empty blocks on top, with many miners building on top of the previous invalid blocks, all of which were not accepted by the network.

So to answer your question:

> So why keep mining on top of the previous block?

Because the new one may be invalid. It's unlikely to happen intentionally due to the costs of mining an invalid block, but it can happen as a result of software or manual bugs. Furthermore, we should not build infrastructure that relies on this not happening - as doing so might over time make such attacks cheaper.

> How long does this process typically take? What does it depend on?

In your process you're only counting block validation. But the whole process consists of everything between (A) the creation of a valid block on the network and (B) hashers switching to building a block on top of it. This includes:

* The previous block creator *getting the block out to the network*. There may be unintentional delays here, or even intentional delays (like a Selfish Mining attack).
* The blocks needs to *propagate across the network*. Normal `bitcoind` nodes only propagate after full validation, and require bursts of high bandwidth to transfer all blocks. More recent technology like Compact Blocks (BIP152) and FIBRE avoid full resubmission or even waiting until validation completed.
* The blocks need to be *validated*.
* A new set of valid *transactions on top* has to be created.

Validation depends on many factors:

* *Software version*. There are constantly improvements to validation speed.
* *UTXO cache size*. The larger the cache, the less database access is needed to retrieve information about the outputs being spent. As a result, just fetching of inputs can take from a few milliseconds to a few seconds.
* *Signature cache size and CPU speed*. The larger the cache, the more signature validations can be avoided. These validations - depending on the software version and hardware can vary from 0.01ms to 0.6ms per signature (45ms to 2.7s for a block).
* *Correlation between the memory pool and the new block*. If a block contains transactions that a node hasn't seen before, its inputs and signatures are less likely to be cached before.
* *Bandwidth*. Prior to Bitcoin Core 0.13, blocks were always submitted in full between peers, which can cause big spikes at the time a new block is announced.
* *Network latency*. In more isolated parts of the world, even with good bandwidth, the time it takes for a network packet to reach the outside world can be significant. Depending on the protocol, 1 to 3 roundtrips are needed to send a block. If the latency between two peers is 200ms, that can already mean 1.2s wasted on going back and forth.
* *Number of connections*. If a node has many connections, it will try to broadcast a new block simultaneously to all, causing a spike of work of network activity. This can be too much for the CPU or network hardware or bandwidth to handle, causing slow downs when many connections exists.

The time to construct a new block mostly depends on software version. In older versions it was up to a few seconds, but lately is has been reduced to tens of milliseconds.

> Is it done on specialized mining hardware or on a general-purpose CPU?

As far as I know, no custom hardware exists for block validation or block construction.

> Is it correct that the time block validation takes is linear to the block's + witness's size should segwit activate?

Mostly. There exists an inefficiency in the currently signature hashing algorithm that can be O(n^2) in the size of transactions. This can result in single transactions that take minutes to just compute the signature hash. This is fixed in BIP144, which is always used inside SegWit transaction inputs, making it O(n) at worst (less than 10ms for a block in the worst case on common hardware).

Longer term, there are other factors that play, like the size of the UTXO set. If this would grow to multiple gigabytes, and not fit in typical memory caches, the UTXO fetch time for validation could go up dramatically.

> Also: How long does it take on average for a block to be propagated through the network? It'd be great if that average is weighted on the receiving miners' hashing powers.

That's complicated. It's certainly not proportional to hashing speed, but more related to network topology and used technology. The FIBRE website has some statistics, but often transfers in less than 20ms more than the minimum theoretical network latency (speed of light across long connections) across the globe. This is only possible by being a private network that assumes its participants will not engage in DoS attacks on the network. The public network is much more robust, but often takes many seconds to propagate a block to large portions of nodes, and dozens of seconds to reach less connected nodes.

> **Comments:**
> **username_0**: Do you know whether some kind of slow loris attack is possible? That'd be the miner of a block sending a block header to their peers so they know there is a new block but then taking really long to send the rest of the block. I expect there to be maximum time span until a block is fully received. Do you know it? Furthermore, this begs the quesiton of whether a miner who found a block without fully having received the previous one is able to send their new block to their peers. Will they accept the newly mined block even if they don't have its parent either?  
> **username_4**: That's probably possible (I don't know the exact details of these validationless mining setups), and it would be a good reason not to do validationless mining.  

---

### Answer by username_2 on 2016-12-28. Score: 2

I think all big miners have own developed software to manage mining process and able to switch to recently received block header before validation is complete. Block validation process should not take a lot of time, because most of transactions from block already in mempool and already verified. Validation task is verify missed transactions, verify coinbase, verify block reward, verify merkleroot. So block validation should complete in few seconds.
Block to be propagated through the network, depends on connection speed.


---

### Answer by username_3 on 2016-12-28. Score: 2

As stated above many miners today already start mining using only the new blockheader before the block is validated and downloaded.

This is why you sometimes see newly generated block with only one transaction (the coinbase transaction). An example of this is the following block:
<https://blockchain.info/block/00000000000000000a06dbd18a15a452c4dd50f662044e654f83066da2775ed8>

This is because the miner does not know exactly which transactions were included in the previous block before downloading and validating it. Because of this, it only includes the coinbase transaction until the mempool has been updated to avoid including a transaction that has already been included in the last block.


---

