<p>A HD wallet doesn’t know what indexes of a seed to scan for until the ones prior have been used, which can’t be discovered out of order. To allow for out of order scanning the address gap limit would need to be prohibitively large, and might still result in re-scanning filters if the wallet is particularly active.</p>

CC BY-SA 4.0
2020-12-29T10:32:40.437Z
Hmm, but I was under the impression that the wallet would have to scan up to the gap-limit anyway, in case some of the indexes end up being "skipped", eg. 3 addresses are generated, but only the 3rd ends up being used.
cloudhead
CC BY-SA 4.0
2020-12-29T10:53:23.377Z
Thinking about this more, it makes sense if what you mean is that we need to scan up to `highest_used_index + gap_limit`, such that everytime a higher index is used, we scan further. Is that the idea?
cloudhead
CC BY-SA 4.0
2020-12-29T11:30:26.133Z
Yes, that's exactly the point: the address body is expanded as time progresses and you might not have derived sufficient addresses to find all your transactions if you start with later blocks. On the other hand, if you just set an enormous gap limit, scanning the blocks for matches will take forever.
Murch
CC BY-SA 4.0
2020-12-29T11:48:32.880Z
Especially in the case of systems like BIP157, having a large gap limit just reduces inefficiency as the filters get dirtier unnecessarily. Scanning them only sequentially isn't of particular concern regardless as they tend to be processed rather quickly once received (or cached).
Claris
CC BY-SA 4.0
2020-12-29T10:21:44.143Z
2020-12-29T11:51:01.563Z
2020-12-29T11:51:01.563Z
Claris
<p>In <a href="https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki" rel="noreferrer">BIP157</a>, under &quot;Client Operation&quot;, we have the following paragraph:</p>
<blockquote>
<p>Starting from the first block in the desired range, the client now MAY download the filters. The client SHOULD test that each filter links to its corresponding filter header and ban peers that send incorrect filters. The client MAY download multiple filters at once to increase throughput, <strong>though it SHOULD test the filters sequentially</strong>. [...]</p>
</blockquote>
<p>I'm wondering about the bolded part: why should the client test the filters in sequence? Why can't it test them as it receives them? I'm having a hard time coming up with a good reason to test them sequentially. Anyone knows?</p>

<wallet><development><protocol><compact-client-side-block-filters>
Why should BIP157 compact filters be processed in-order?
cloudhead
# Why should BIP157 compact filters be processed in-order?

(Asked by: username_0 on 2020-12-29)

In [BIP157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki), under "Client Operation", we have the following paragraph:

> Starting from the first block in the desired range, the client now MAY download the filters. The client SHOULD test that each filter links to its corresponding filter header and ban peers that send incorrect filters. The client MAY download multiple filters at once to increase throughput, **though it SHOULD test the filters sequentially**. [...]

I'm wondering about the bolded part: why should the client test the filters in sequence? Why can't it test them as it receives them? I'm having a hard time coming up with a good reason to test them sequentially. Anyone knows?

**Tags:** wallet, development, protocol, compact-client-side-block-filters


---

## 1 answers

### Answer by username_1 on 2020-12-29. Score: 4 | Accepted answer

A HD wallet doesn’t know what indexes of a seed to scan for until the ones prior have been used, which can’t be discovered out of order. To allow for out of order scanning the address gap limit would need to be prohibitively large, and might still result in re-scanning filters if the wallet is particularly active.

> **Comments:**
> **username_0**: Hmm, but I was under the impression that the wallet would have to scan up to the gap-limit anyway, in case some of the indexes end up being "skipped", eg. 3 addresses are generated, but only the 3rd ends up being used.  
> **username_0**: Thinking about this more, it makes sense if what you mean is that we need to scan up to `highest_used_index + gap_limit`, such that everytime a higher index is used, we scan further. Is that the idea?  
> **username_2**: Yes, that's exactly the point: the address body is expanded as time progresses and you might not have derived sufficient addresses to find all your transactions if you start with later blocks. On the other hand, if you just set an enormous gap limit, scanning the blocks for matches will take forever.  
> **username_1**: Especially in the case of systems like BIP157, having a large gap limit just reduces inefficiency as the filters get dirtier unnecessarily. Scanning them only sequentially isn't of particular concern regardless as they tend to be processed rather quickly once received (or cached).  

---

