<blockquote>
<p>Is it that blocks aren't in order in blk files ?</p>
</blockquote>
<p>This is true. For example, during the initial-block-loading phase, your node may be <em>concurrently</em> collecting different ranges of blocks from multiple peers. So they can arrive out of order. You also have dead forks.</p>
<p>As Murch noted: &quot;the .blk files store block data in the order that it is received by the node, rather than in the order it was requested or the order of their heights.&quot;</p>
<p>So nodes, or other programs processing the block data files probably all need to create indexes of blocks by block-ID and height as well as indexes of transactions by transaction-ID etc. Some indexes might be of all blocks and others might be only of blocks in the currently active chain.</p>
<p>I can't tell whether the above accounts for the whole of your issue.</p>

CC BY-SA 4.0
2022-09-02T11:03:26.487Z
Thanks for the explanation, so theoritically I have to parse and hash every block. And then looking into another block if the hash matches starting from block with height 0 ?
But dead forks aren't on the main net are they ? 


And thanks for the python tip :D ^^
LuckyAutomata
CC BY-SA 4.0
2022-09-02T11:59:03.780Z
Yes, you need to do something like build an index of blocks with the block hash as key. Since occasionally two miners produce blocks at the same height, your BLK files will contain both even after one is selected as constituting part of the main chain and the other marked stale.
RedGrittyBrick
CC BY-SA 4.0
2022-09-07T19:00:23.573Z
It's in there, but I feel it could be perhaps stated a bit more clearly that the `.blk` files store block data in the order that it is received by the node, rather than in the order it was requested or the order of their heights.
Murch
CC BY-SA 4.0
2022-09-02T10:33:06.170Z
2022-10-06T09:31:56.320Z
2022-10-06T09:31:56.320Z
RedGrittyBrick
<p>Hello I'm using a python script to actually retrieve information from a blk file and to read data.</p>
<pre><code>import datetime

# Read block !
file = open(&quot;REDACTED\blocks\\blk00000.dat&quot;, 'rb')

numberOfBlocks = 0
i = 0
toAnalyse = 5000

while i &lt; toAnalyse:  

    block_magic_bytes = file.read(4)
    
    if b''==block_magic_bytes :
        break
    
    block_size = file.read(4)
    block_size_int = int.from_bytes(block_size,'little')
    block_version = file.read(4)
    block_lastblock = file.read(32)
    block_lastblock_little_endian = int.from_bytes(block_lastblock,'little') 

    # merkle root
    file.seek(32,1)

    block_time = int.from_bytes(file.read(4),'little')

    #bits
    file.seek(4,1)

    block_nonce = int.from_bytes(file.read(4),'little')

    # Prints some information
    print(&quot;Block number   : &quot; + str(numberOfBlocks))
    print(numberOfBlocks)

    print(&quot;Magic bytes    : &quot; + block_magic_bytes.hex(&quot; &quot;).upper())
    print(&quot;Block size hex : &quot; + block_size.hex(&quot; &quot;).upper())
    print(&quot;Block size int : &quot; + str(block_size_int))
    print(&quot;Block version  : &quot; + block_version.hex(&quot; &quot;))
    print( f'Block preBlock : {block_lastblock_little_endian:064x}' )
    print(&quot;Block time     : &quot; + str(datetime.datetime.fromtimestamp(block_time)))
    print(&quot;Block nonce    : &quot; + str(block_nonce))
    print()

    # Jumps to next magic_bytes
    file.seek(block_size_int-4-32-32-4-4-4,1)

    numberOfBlocks+=1
    i+=1
print(&quot;Last block read&quot;)

</code></pre>
<p>But when I read the block 486 in the blk file I get this :</p>
<pre><code>Block number   : 486
486
Magic bytes    : F9 BE B4 D9
Block size hex : D8 00 00 00
Block size int : 216
Block version  : 01 00 00 00
Block preBlock : 00000000806df68baab17e49e567d4211177fef4849ffd8242d095c6a1169f45
Block time     : 2009-01-14 22:27:29
Block nonce    : 2063568441
</code></pre>
<p>As I understand the BTC blockchain the previous hash should be the hash of the block with height 485, but when I look using an explorer this is the hash of block 499 : <a href="https://www.blockchain.com/btc/block/00000000806df68baab17e49e567d4211177fef4849ffd8242d095c6a1169f45" rel="nofollow noreferrer">https://www.blockchain.com/btc/block/00000000806df68baab17e49e567d4211177fef4849ffd8242d095c6a1169f45</a></p>
<p>Is it that blocks aren't in order in blk files ? Are my blk files wrong ? I downloaded them using bitcoin-core. Are some blocks actually not included in the overall blockchain ?</p>
<p>I would appreciate any pointers,tips or help of course ^^</p>

<blockchain><block><python>
Wrong hashes for blocks when parsing from blk
LuckyAutomata
# Wrong hashes for blocks when parsing from blk

(Asked by: username_0 on 2022-09-02)

Hello I'm using a python script to actually retrieve information from a blk file and to read data.

```
import datetime

# Read block !
file = open("REDACTED\blocks\\blk00000.dat", 'rb')

numberOfBlocks = 0
i = 0
toAnalyse = 5000

while i < toAnalyse:  

    block_magic_bytes = file.read(4)
    
    if b''==block_magic_bytes :
        break
    
    block_size = file.read(4)
    block_size_int = int.from_bytes(block_size,'little')
    block_version = file.read(4)
    block_lastblock = file.read(32)
    block_lastblock_little_endian = int.from_bytes(block_lastblock,'little') 

    # merkle root
    file.seek(32,1)

    block_time = int.from_bytes(file.read(4),'little')

    #bits
    file.seek(4,1)

    block_nonce = int.from_bytes(file.read(4),'little')

    # Prints some information
    print("Block number   : " + str(numberOfBlocks))
    print(numberOfBlocks)

    print("Magic bytes    : " + block_magic_bytes.hex(" ").upper())
    print("Block size hex : " + block_size.hex(" ").upper())
    print("Block size int : " + str(block_size_int))
    print("Block version  : " + block_version.hex(" "))
    print( f'Block preBlock : {block_lastblock_little_endian:064x}' )
    print("Block time     : " + str(datetime.datetime.fromtimestamp(block_time)))
    print("Block nonce    : " + str(block_nonce))
    print()

    # Jumps to next magic_bytes
    file.seek(block_size_int-4-32-32-4-4-4,1)

    numberOfBlocks+=1
    i+=1
print("Last block read")

```

But when I read the block 486 in the blk file I get this :

```
Block number   : 486
486
Magic bytes    : F9 BE B4 D9
Block size hex : D8 00 00 00
Block size int : 216
Block version  : 01 00 00 00
Block preBlock : 00000000806df68baab17e49e567d4211177fef4849ffd8242d095c6a1169f45
Block time     : 2009-01-14 22:27:29
Block nonce    : 2063568441

```

As I understand the BTC blockchain the previous hash should be the hash of the block with height 485, but when I look using an explorer this is the hash of block 499 : <https://www.blockchain.com/btc/block/00000000806df68baab17e49e567d4211177fef4849ffd8242d095c6a1169f45>

Is it that blocks aren't in order in blk files ? Are my blk files wrong ? I downloaded them using bitcoin-core. Are some blocks actually not included in the overall blockchain ?

I would appreciate any pointers,tips or help of course ^^

**Tags:** blockchain, block, python


---

## 1 answers

### Answer by username_1 on 2022-09-02. Score: 0

> Is it that blocks aren't in order in blk files ?

This is true. For example, during the initial-block-loading phase, your node may be *concurrently* collecting different ranges of blocks from multiple peers. So they can arrive out of order. You also have dead forks.

As username_2 noted: "the .blk files store block data in the order that it is received by the node, rather than in the order it was requested or the order of their heights."

So nodes, or other programs processing the block data files probably all need to create indexes of blocks by block-ID and height as well as indexes of transactions by transaction-ID etc. Some indexes might be of all blocks and others might be only of blocks in the currently active chain.

I can't tell whether the above accounts for the whole of your issue.

> **Comments:**
> **username_0**: Thanks for the explanation, so theoritically I have to parse and hash every block. And then looking into another block if the hash matches starting from block with height 0 ? But dead forks aren't on the main net are they ?    And thanks for the python tip :D ^^  
> **username_1**: Yes, you need to do something like build an index of blocks with the block hash as key. Since occasionally two miners produce blocks at the same height, your BLK files will contain both even after one is selected as constituting part of the main chain and the other marked stale.  
> **username_2**: It's in there, but I feel it could be perhaps stated a bit more clearly that the `.blk` files store block data in the order that it is received by the node, rather than in the order it was requested or the order of their heights.  

---

