<p>It is perfectly possible that *.blk files contains gaps of zeroes, or even partially written blocks. In modern versions (since 0.10 or so) this won't happen during normal operations, but it can't be prevented in case of abnormal shutdown (crash, out of disk space, ...).</p>

<p>If you want to be sure, you need to scan the entire file for potential blocks (they'll always start with a magic + length descriptor, so you can quickly skip many things). That, or use the getblock RPC instead, which uses Bitcoin Core's prebuilt index.</p>

CC BY-SA 3.0
2016-11-13T19:39:19.697Z
Thanks again Pieter. I can understand parially written blocks, but what would be the reason for the presence of gaps or zeroes in a *.blk file?
inersha
CC BY-SA 3.0
2016-11-13T19:02:03.403Z
2016-11-13T19:02:03.403Z
Pieter Wuille
<p>I've been parsing the <strong>blk*.dat</strong> files using <code>fread()</code>.</p>

<p>Let's say this is the basic structure of a blk.dat file:</p>

<pre><code>[magicbyte] [blockheader] [transactions]
[magicbyte] [blockheader] [transactions]
...
[magicbyte] [blockheader] [transactions]
</code></pre>

<p>When parsing, I start by reading the first 4 bytes.</p>

<ul>
<li>If it's equal to <code>D9B4BEF9</code> (magicbyte), I gather the [blockheader] and [transactions] data.</li>
<li>If I get <code>NULL</code>, I assume that I've reached the end of the blk.dat file and move on to the next one (if there is one).</li>
<li>If I get <code>00000000</code>, I assume that the blk.dat file is incomplete, and I keep re-reading it.</li>
</ul>

<p>Is this a reliable way to parse the blk.dat files? The <code>00000000</code> check is what I'm most unsure of.</p>

<raw-data>
How can you tell if you're at the end of an incomplete blk*.dat file?
inersha
# How can you tell if you're at the end of an incomplete blk*.dat file?

(Asked by: username_0 on 2016-11-13)

I've been parsing the **blk\*.dat** files using `fread()`.

Let's say this is the basic structure of a blk.dat file:

```
[magicbyte] [blockheader] [transactions]
[magicbyte] [blockheader] [transactions]
...
[magicbyte] [blockheader] [transactions]

```

When parsing, I start by reading the first 4 bytes.

* If it's equal to `D9B4BEF9` (magicbyte), I gather the [blockheader] and [transactions] data.
* If I get `NULL`, I assume that I've reached the end of the blk.dat file and move on to the next one (if there is one).
* If I get `00000000`, I assume that the blk.dat file is incomplete, and I keep re-reading it.

Is this a reliable way to parse the blk.dat files? The `00000000` check is what I'm most unsure of.

**Tags:** raw-data


---

## 1 answers

### Answer by username_1 on 2016-11-13. Score: 2

It is perfectly possible that \*.blk files contains gaps of zeroes, or even partially written blocks. In modern versions (since 0.10 or so) this won't happen during normal operations, but it can't be prevented in case of abnormal shutdown (crash, out of disk space, ...).

If you want to be sure, you need to scan the entire file for potential blocks (they'll always start with a magic + length descriptor, so you can quickly skip many things). That, or use the getblock RPC instead, which uses Bitcoin Core's prebuilt index.

> **Comments:**
> **username_0**: Thanks again username_1. I can understand parially written blocks, but what would be the reason for the presence of gaps or zeroes in a *.blk file?  

---

