<p>Not all wallets are the same, but a good wallet should always follow standards. A standard for address-generation generally accepted is BIP44. In this way, you could scan the first consecutive addresses that a BIP44-compliant wallet would create. If the wallet supports Segwit, then you should look at BIP49 and BIP84 as well.</p>
<p>As Raghav Sood pointed out in his answer, most wallets scan a certain amount of unused addresses before they decide to stop scanning for funds. Most of them use 20 as the gap limit (maximum gap in between unused addresses).</p>
<p>Now, to answer your question, you are totally right. Nothing prevents a wallet from not complying with BIP44 standard or with any other standard whatsoever. That is why it is important to know the wallet you are dealing with. However, a well defined protocol/standard helps with the fund-discovery process which is super important for any kind of wallet, for which it is almost certain that every wallet out there possesses a standard for address creation (either BIP44 or one of their own).</p>
<p>Just find out the standard that the wallet you're dealing with is following for address creation, and that would lead you to find the most efficient way to find the UTXO associated to the xpub.</p>

CC BY-SA 4.0
2020-09-09T13:22:30.907Z
Thing is the software being built is about tracking funds from any source the xpub is from without discrimination. I wanted to know if there is a sure way to track all funds without making billions of requests. What I have come to realize is that if a wallet goes beyond the 20 empty derivation gap, some actual wallets might not keep track of those extra UTXOs as we speak.
RooSoft
CC BY-SA 4.0
2020-09-09T13:59:50.807Z
ah ok. Then, I guess the standards, as I mentioned, would work for most cases. And yes, you are right about the gap. Although, you are usually able to fix that number if you notice that the total of your funds is not being reflected. Take into account also the mechanics of wallets for address creation, and what are the odds of an address to be used beyond the 20 gap. In your case, if you want to make sure that you are scanning the total of the funds, then you can use a wider gap. It might seem too much, but that is the most efficient way for scanning. There is no other possible shortcut.
Oscar Serna
CC BY-SA 4.0
2020-09-09T14:10:26.883Z
Thank you so much Oscar... helps increase confidence I have in what I am trying to do.
RooSoft
CC BY-SA 4.0
2020-09-09T04:41:37.137Z
2020-09-09T04:52:44.413Z
2020-09-09T04:52:44.413Z
Oscar Serna
<p>The balance checks are cut short based on a gap limit.</p>
<p>A gap limit is usually set to 20, and is defined as a consecutive block of addresses with no transactions.</p>
<p>Since HD wallets only give out a new address on explicit request, or automatically when the previously issued address has been used, it is assumed that someone will not leave a block of 20 consecutive addresses unused.</p>
<p>Wallet software will generate a short list of maybe addresses when you load an xpub or other form of seed. It will then check through that, and if it doesn't find 20 consecutive unused addresses, it will generate another short list (so the first list might cover 0-100, then 100-200, etc). until it finds the block of unused addresses.</p>
<p>This assumption naturally falls apart if you are running an exchange, payment service, or other wallet set up where it is very likely that you will have many hundreds of unused addresses. Those systems generally keep track of assigned addresses out of band, and have specialized indexer systems that can track many millions of addresses.</p>

CC BY-SA 4.0
2020-09-09T12:55:07.770Z
The software I am building is about keeping track of funds related to an xpub/ypub/zpub no matter the origin. My questioning is about the fact that nobody can prevent wallets from overriding the gap limit. Anybody can write one pretty easily. So I guess there is no totally reliable way to know how many funds are related to an xpub.
RooSoft
CC BY-SA 4.0
2020-09-10T03:01:19.603Z
If you intend to deal with arbitrary xpubs, then yes, you have no choice but to check every address on every possible account in that xpub, which is not a viable approach (a single xpub will lead to many trillions of addresses). If you are only dealing with xpubs generated by other, sane wallets like electrum, ledger, trezor, etc. then the gap limit is a safe assumption
Raghav Sood
CC BY-SA 4.0
2020-09-10T03:02:34.583Z
RooSoft, You're thinking of an xpub like an address, which it is not. It is a brute force issue if someone uses a random address deep in the chain. Fortunately no wallet software will use a random index so utilizing the gap limit approach is the most efficient way to check for UTXOs related to an xpub.
m1xolyd1an
CC BY-SA 4.0
2020-09-09T02:47:04.410Z
2020-09-09T02:47:04.410Z
Raghav Sood
<p>Apparently the way to find a list of UTXO related to an xpub is to derive all addresses from the master private key and query a node with each and every one of them.</p>
<p>It is possible to generate <code>4294967296</code> addresses from a <em>single extended key</em>.</p>
<p>How should a wallet proceed to manage this in an efficient way?</p>
<p>I guess that in an ideal situation, addresses are generated in a sequential order... but my intuition tells me that nothing prevents a wallet from doing otherwise.</p>

<utxo><xpub>
Best way to find list of UTXO related to xpub
RooSoft
# Best way to find list of UTXO related to xpub

(Asked by: username_0 on 2020-09-08)

Apparently the way to find a list of UTXO related to an xpub is to derive all addresses from the master private key and query a node with each and every one of them.

It is possible to generate `4294967296` addresses from a *single extended key*.

How should a wallet proceed to manage this in an efficient way?

I guess that in an ideal situation, addresses are generated in a sequential order... but my intuition tells me that nothing prevents a wallet from doing otherwise.

**Tags:** utxo, xpub


---

## 2 answers

### Answer by username_2 on 2020-09-09. Score: 7 | Accepted answer

The balance checks are cut short based on a gap limit.

A gap limit is usually set to 20, and is defined as a consecutive block of addresses with no transactions.

Since HD wallets only give out a new address on explicit request, or automatically when the previously issued address has been used, it is assumed that someone will not leave a block of 20 consecutive addresses unused.

Wallet software will generate a short list of maybe addresses when you load an xpub or other form of seed. It will then check through that, and if it doesn't find 20 consecutive unused addresses, it will generate another short list (so the first list might cover 0-100, then 100-200, etc). until it finds the block of unused addresses.

This assumption naturally falls apart if you are running an exchange, payment service, or other wallet set up where it is very likely that you will have many hundreds of unused addresses. Those systems generally keep track of assigned addresses out of band, and have specialized indexer systems that can track many millions of addresses.

> **Comments:**
> **username_0**: The software I am building is about keeping track of funds related to an xpub/ypub/zpub no matter the origin. My questioning is about the fact that nobody can prevent wallets from overriding the gap limit. Anybody can write one pretty easily. So I guess there is no totally reliable way to know how many funds are related to an xpub.  
> **username_2**: If you intend to deal with arbitrary xpubs, then yes, you have no choice but to check every address on every possible account in that xpub, which is not a viable approach (a single xpub will lead to many trillions of addresses). If you are only dealing with xpubs generated by other, sane wallets like electrum, ledger, trezor, etc. then the gap limit is a safe assumption  
> **username_3**: username_0, You're thinking of an xpub like an address, which it is not. It is a brute force issue if someone uses a random address deep in the chain. Fortunately no wallet software will use a random index so utilizing the gap limit approach is the most efficient way to check for UTXOs related to an xpub.  

---

### Answer by username_1 on 2020-09-09. Score: 1

Not all wallets are the same, but a good wallet should always follow standards. A standard for address-generation generally accepted is BIP44. In this way, you could scan the first consecutive addresses that a BIP44-compliant wallet would create. If the wallet supports Segwit, then you should look at BIP49 and BIP84 as well.

As username_2 username_2 pointed out in his answer, most wallets scan a certain amount of unused addresses before they decide to stop scanning for funds. Most of them use 20 as the gap limit (maximum gap in between unused addresses).

Now, to answer your question, you are totally right. Nothing prevents a wallet from not complying with BIP44 standard or with any other standard whatsoever. That is why it is important to know the wallet you are dealing with. However, a well defined protocol/standard helps with the fund-discovery process which is super important for any kind of wallet, for which it is almost certain that every wallet out there possesses a standard for address creation (either BIP44 or one of their own).

Just find out the standard that the wallet you're dealing with is following for address creation, and that would lead you to find the most efficient way to find the UTXO associated to the xpub.

> **Comments:**
> **username_0**: Thing is the software being built is about tracking funds from any source the xpub is from without discrimination. I wanted to know if there is a sure way to track all funds without making billions of requests. What I have come to realize is that if a wallet goes beyond the 20 empty derivation gap, some actual wallets might not keep track of those extra UTXOs as we speak.  
> **username_1**: ah ok. Then, I guess the standards, as I mentioned, would work for most cases. And yes, you are right about the gap. Although, you are usually able to fix that number if you notice that the total of your funds is not being reflected. Take into account also the mechanics of wallets for address creation, and what are the odds of an address to be used beyond the 20 gap. In your case, if you want to make sure that you are scanning the total of the funds, then you can use a wider gap. It might seem too much, but that is the most efficient way for scanning. There is no other possible shortcut.  
> **username_0**: Thank you so much username_1... helps increase confidence I have in what I am trying to do.  

---

