<p>The keypool is ordered, not just a set of keys being watched. This allows for addresses to be given out in derivation order, as well as maintaining the gap limit. When an address is used, all keys that come before it will also be marked as used. This is done to ensure that already used addresses are not accidentally reused.</p>
<p>So when you give out and use a random address in the range, you end up having all of the previous keys in the range be marked as used.</p>
<p>Removing keys from the keypool is purely for giving addresses out to the user and to be able to generate further addresses. It does not effect the watching behavior; keys removed from the keypool will still be watched.</p>
<p>The reason that you see a performance hit is because removing from the keypool deletes a record from the database in addition to writing several scripst to the database. Furthermore, if the keypool size drops below the configured keypool size due to these removals, the keypool will be refilled which will involve additional database writes. These all acquire a lock on the wallet which prevents other threads from accessing the wallet, and writing all of this data (especially tens of thousands of keys) can take a long time.</p>
<p>To avoid this problem, I highly suggest giving out addresses in order, not randomly. Otherwise you may not want to add the keys to the keypool although that means that you will lose the keypool refilling to keep the range full.</p>

CC BY-SA 4.0
2020-11-07T02:46:10.017Z
Makes a lot of sense, thanks for the answer. I should definitely not be adding them to the keypool for this specific use case.
Fontaine
CC BY-SA 4.0
2024-03-01T21:41:03.047Z
This is so helpful thanks for this answer
Shovas
CC BY-SA 4.0
2020-11-06T02:40:36.387Z
2020-11-06T02:40:36.387Z
Ava Chow
<p>I opened my laptop which always has bitcoind running and my log is going ballistic with messages like:</p>
<p><code>2020-11-05T12:41:57Z [FullyNoded-d703055f46595edcf8163916bd906f9563afc0903fbe74c2882a93c59359ca00] keypool index 38212 removed </code></p>
<p>Why? What is it doing? Removing keys I imported? I never asked it to do that :(</p>
<p>RPC commands are also unresponsive as it seems to be overwhelmed.</p>
<p>Bitcoin Core 0.20.1</p>

CC BY-SA 4.0
2020-11-05T16:48:21.900Z
Are these occurring in order (e.g. 38211 removed, then 38212 removed)? Did you retrieve a lot of addresses but only receive Bitcoin at one that came after a lot of addresses?
Ava Chow
CC BY-SA 4.0
2020-11-06T01:06:26.910Z
yes and yes. Im watching 99,999 addresses and in order to generate payment addresses I randomly generate an address from range 0...99999 for a customer to pay. Does this log output mean it is no longer going to `watch` for those addresses?
Fontaine
CC BY-SA 4.0
2020-11-06T01:29:22.803Z
fwiw the keypool is not relevant for my use case, all that matters is that bitcoind is watching those addresses and that I can use `deriveaddresses`.
Fontaine
<bitcoind><keypool>
Why is bitcoind removing keys from keypool?
Fontaine
# Why is bitcoind removing keys from keypool?

(Asked by: username_0 on 2020-11-05)

I opened my laptop which always has bitcoind running and my log is going ballistic with messages like:

`2020-11-05T12:41:57Z [FullyNoded-d703055f46595edcf8163916bd906f9563afc0903fbe74c2882a93c59359ca00] keypool index 38212 removed`

Why? What is it doing? Removing keys I imported? I never asked it to do that :(

RPC commands are also unresponsive as it seems to be overwhelmed.

Bitcoin Core 0.20.1

**Tags:** bitcoind, keypool

> **Comments:**
> **username_1**: Are these occurring in order (e.g. 38211 removed, then 38212 removed)? Did you retrieve a lot of addresses but only receive Bitcoin at one that came after a lot of addresses?  
> **username_0**: yes and yes. Im watching 99,999 addresses and in order to generate payment addresses I randomly generate an address from range 0...99999 for a customer to pay. Does this log output mean it is no longer going to `watch` for those addresses?  
> **username_0**: fwiw the keypool is not relevant for my use case, all that matters is that bitcoind is watching those addresses and that I can use `deriveaddresses`.  

---

## 1 answers

### Answer by username_1 on 2020-11-06. Score: 3 | Accepted answer

The keypool is ordered, not just a set of keys being watched. This allows for addresses to be given out in derivation order, as well as maintaining the gap limit. When an address is used, all keys that come before it will also be marked as used. This is done to ensure that already used addresses are not accidentally reused.

So when you give out and use a random address in the range, you end up having all of the previous keys in the range be marked as used.

Removing keys from the keypool is purely for giving addresses out to the user and to be able to generate further addresses. It does not effect the watching behavior; keys removed from the keypool will still be watched.

The reason that you see a performance hit is because removing from the keypool deletes a record from the database in addition to writing several scripst to the database. Furthermore, if the keypool size drops below the configured keypool size due to these removals, the keypool will be refilled which will involve additional database writes. These all acquire a lock on the wallet which prevents other threads from accessing the wallet, and writing all of this data (especially tens of thousands of keys) can take a long time.

To avoid this problem, I highly suggest giving out addresses in order, not randomly. Otherwise you may not want to add the keys to the keypool although that means that you will lose the keypool refilling to keep the range full.

> **Comments:**
> **username_0**: Makes a lot of sense, thanks for the answer. I should definitely not be adding them to the keypool for this specific use case.  
> **username_2**: This is so helpful thanks for this answer  

---

