<p>There is a lot of information available, just not consolidated. After searching for sometime, I found the best answer from <a href="http://%20%20%20%204%20bytes:%20Encodes%20size%20N%20of%20next%20upcoming%20block.%20%20%20%20%20N%20bytes:%20Block%20encoded%20in%20standard%20format%20(with%20header)" rel="nofollow noreferrer">this link</a>.</p>

<p>The format of each file is:</p>

<pre><code>4 bytes: Magic bytes
4 bytes: Encodes size N of next upcoming block.
N bytes: Block encoded in standard format (with header)
--
above repeats
</code></pre>

<p>Quotes from the original link:</p>

<pre><code>4 | 4 | 80 | TxData | 4 | 4 | 80 | TxData | 4 | 4 | 80 | TxData | ...
</code></pre>

<ul>
<li>First 4 bytes: magic bytes (identifying which network you are on)</li>
<li>Second 4 bytes: the number of bytes of the remaining block</li>
<li>Next 80 bytes: block header itself</li>
<li>Next NumBlockBytes - 80 bytes: Transaction data in this block [ numTx | Tx1 | Tx2 | Tx3 | ... ]</li>
</ul>

<p>What I don't understand is why are the magic bytes repeated? We could very well have used:</p>

<pre><code>4 | 4 | 80 | TxData | 4 | 80 | TxData | 4 | 80 | TxData | ...
</code></pre>

CC BY-SA 3.0
2017-12-13T07:56:50.463Z
>>> What I don't understand is why are the magic bytes repeated? <<<< because Core programmers use **the same** serialization routine for transferring blocks by network and storing on disk. You can store blocks on your own hard-drive in **any** format. **Nobody cares.**
amaclin
CC BY-SA 3.0
2017-12-14T14:17:35.883Z
Yup, makes sense. I was wondering, if some space could be saved if bitcoind didn't store those bytes.
Jus12
CC BY-SA 3.0
2017-12-13T06:35:01.610Z
2017-12-13T07:16:46.817Z
2017-12-13T07:16:46.817Z
Jus12
<blockquote>
  <p>I want to parse blockchain manually</p>
</blockquote>

<p>If you mean <code>blockchain</code> is a RAW on-disk data, which contains all information about the blocks and transactions (../blocks/blk*.dat files), so you can try <a href="https://github.com/normanvolt/blockchain-parser" rel="nofollow noreferrer">blockchain parser</a>. This is a standalone script that can parse that files. No installation needed, no dependencies.</p>

<blockquote>
  <p>Is there any way to understand how the blocks are stored without going through the C++ code?</p>
</blockquote>

<p>The blockchain parser by the link can give you the way to understand how the bocks are stored in a simple intuitive view.</p>

<blockquote>
  <p>extract the UTXO set</p>
</blockquote>

<p>For extraction of the UTXOs (for constructing UTXO set) you need to understand that this data did not stored permanently on disk.</p>

<p>There are two ways for building the UTXO set:</p>

<ol>
<li>Parse the chainstate frozen dump (../chainstate/*.ldb files)</li>
<li>Construct UTXO set from parsed RAW data (../blocks/blk*.dat files)</li>
</ol>

<p>The first way is faster, but not so usefull for deep understanding of how blockchain really works.
The second way is what I'am trying to do now. This also called <code>blockchain verification</code> by third-parties scripting. I made the script for UTXO set building from RAW dumps, but my implementation on Python is very very slow.</p>

<p>I hope my answer may help you in your explorations.</p>

CC BY-SA 4.0
2018-06-27T06:01:35.843Z
2018-06-27T06:01:35.843Z
Denis Leonov
<p>I want to parse blockchain manually and extract the UTXO set (for various reasons). Is there any way to understand how the blocks are stored without going through the C++ code? </p>

<p>Some specs will be great. </p>

<bitcoind><leveldb>
How does bitcoind store blocks?
Jus12
# How does bitcoind store blocks?

(Asked by: username_0 on 2017-12-12)

I want to parse blockchain manually and extract the UTXO set (for various reasons). Is there any way to understand how the blocks are stored without going through the C++ code?

Some specs will be great.

**Tags:** bitcoind, leveldb


---

## 2 answers

### Answer by username_0 on 2017-12-13. Score: 2 | Accepted answer

There is a lot of information available, just not consolidated. After searching for sometime, I found the best answer from [this link](http://%20%20%20%204%20bytes:%20Encodes%20size%20N%20of%20next%20upcoming%20block.%20%20%20%20%20N%20bytes:%20Block%20encoded%20in%20standard%20format%20(with%20header)).

The format of each file is:

```
4 bytes: Magic bytes
4 bytes: Encodes size N of next upcoming block.
N bytes: Block encoded in standard format (with header)
--
above repeats

```

Quotes from the original link:

```
4 | 4 | 80 | TxData | 4 | 4 | 80 | TxData | 4 | 4 | 80 | TxData | ...

```

* First 4 bytes: magic bytes (identifying which network you are on)
* Second 4 bytes: the number of bytes of the remaining block
* Next 80 bytes: block header itself
* Next NumBlockBytes - 80 bytes: Transaction data in this block [ numTx | Tx1 | Tx2 | Tx3 | ... ]

What I don't understand is why are the magic bytes repeated? We could very well have used:

```
4 | 4 | 80 | TxData | 4 | 80 | TxData | 4 | 80 | TxData | ...

```

> **Comments:**
> **username_1**: >>> What I don't understand is why are the magic bytes repeated? <<<< because Core programmers use **the same** serialization routine for transferring blocks by network and storing on disk. You can store blocks on your own hard-drive in **any** format. **Nobody cares.**  
> **username_0**: Yup, makes sense. I was wondering, if some space could be saved if bitcoind didn't store those bytes.  

---

### Answer by username_2 on 2018-06-27. Score: 0

> I want to parse blockchain manually

If you mean `blockchain` is a RAW on-disk data, which contains all information about the blocks and transactions (../blocks/blk\*.dat files), so you can try [blockchain parser](https://github.com/normanvolt/blockchain-parser). This is a standalone script that can parse that files. No installation needed, no dependencies.

> Is there any way to understand how the blocks are stored without going through the C++ code?

The blockchain parser by the link can give you the way to understand how the bocks are stored in a simple intuitive view.

> extract the UTXO set

For extraction of the UTXOs (for constructing UTXO set) you need to understand that this data did not stored permanently on disk.

There are two ways for building the UTXO set:

1. Parse the chainstate frozen dump (../chainstate/\*.ldb files)
2. Construct UTXO set from parsed RAW data (../blocks/blk\*.dat files)

The first way is faster, but not so usefull for deep understanding of how blockchain really works.
The second way is what I'am trying to do now. This also called `blockchain verification` by third-parties scripting. I made the script for UTXO set building from RAW dumps, but my implementation on Python is very very slow.

I hope my answer may help you in your explorations.


---

