<p>Backwards compatibility is a massive priority for Core and it is generally very conservative in its technology choices. I don't think it would consider a new (database) technology until it had been proven over many years on other projects. It is providing a consensus engine and infrastructure and doesn't have the same tolerance as startups or bigger corporations for cycling through different options and experimentation if it is going to endanger backwards compatibility or require many developer hours to revamp shortly afterwards.</p>

CC BY-SA 4.0
2021-02-12T17:09:50.053Z
2021-02-12T17:09:50.053Z
Michael Folkson
<blockquote>
<p>I can't imagine any other reason as to why BDB wasn't dumped altogether if it wasn't for backwards support.</p>
</blockquote>
<p>There simply is none. The wallet file doesn't need to be a database, it could be a flat text file and that would operate just as well. BDB will remain for compatibility for the foreseeable future, and change is risky, so it remains.</p>

CC BY-SA 4.0
2021-02-12T18:35:49.207Z
Perhaps worth pointing out that BDB is only used for wallet files, which tend to have extremely long lifetimes (just search this site for people trying to get their funds out of wallet.dat files that are a decade old). For other purposes LevelDB is used, and incompatible changes to those databases have been made several times, for performance or other reasons. It's only for the wallet file that it really doesn't matter what format is used (it's loaded entirely into memory, at least for now), and compatibility is paramount.
Pieter Wuille
CC BY-SA 4.0
2021-02-12T17:38:50.650Z
2021-02-12T17:38:50.650Z
Claris
<p>The primary reason Bitcoin's C++ reference implementation uses Berkeley DB 4.8 seems to be <a href="https://github.com/bitcoin/bitcoin/issues/3686#issuecomment-74836708" rel="nofollow noreferrer">backwards compatibility</a>. It doesn't appear that BDB4.8 (or BDB at all for that matter) has any superior advantage over current-day alternatives like Redis or RocksDB, at least based on the limited research I've done (<a href="https://db-engines.com/en/system/LevelDB%3BOracle+Berkeley+DB%3BRocksDB" rel="nofollow noreferrer">source</a>). To the best of my knowledge, most alternative full-node implementations steer away from wallet support (<a href="https://github.com/rust-bitcoin/rust-bitcoin" rel="nofollow noreferrer">source</a>, <a href="https://github.com/MetacoSA/NBitcoin" rel="nofollow noreferrer">source</a>).</p>
<p>Performance benchmarks seem to indicate that there is no performance benefit to sticking to Berkeley DB (<a href="http://highscalability.com/blog/2012/11/29/performance-data-for-leveldb-berkley-db-and-bangdb-for-rando.html" rel="nofollow noreferrer">source</a>) yet I cannot seem to see any widespread adoption of anything else except for recent introduction of SQLite support (<a href="https://github.com/bitcoin/bitcoin/pull/19077" rel="nofollow noreferrer">source</a>) for Bitcoin Core despite the growing pains of BDB4.8. I can't imagine any other reason as to why BDB wasn't dumped altogether if it wasn't for backwards support.</p>
<p>Is there any reason why alternatives like RocksDB (which supports <a href="https://github.com/facebook/rocksdb/blob/master/env/env_encryption.cc" rel="nofollow noreferrer">at rest encryption</a> though it isn't well documented), ArangoDB or Aerospike (all that seem to have existed at least five years ago) were not considered? Is there any security benefit to sticking with BDB? Any redeeming properties of BDB that make it a <em>definitive choice</em> over what is available currently?</p>
<p>Should a new implementation stick to BDB or use another library for wallet support and if so, what should it be?</p>

CC BY-SA 4.0
2021-02-12T16:32:08.530Z
There are a couple of questions here that at least partly answer your question (not the historical decision part though) At least if your needs are similar to the Core wallet today you would probably choose to use sqlite for the wallet rather than BDB https://bitcoin.stackexchange.com/questions/51435/migration-from-berkeley-db-to-leveldb?rq=1 https://bitcoin.stackexchange.com/questions/99620/why-is-the-bitcoin-core-wallet-database-moving-from-berkeley-db-to-sqlite
Michael Folkson
CC BY-SA 4.0
2021-02-12T16:43:12.317Z
I'm familiar with both answers, the first one explains the transition away from BDB for blockchain storage and simply reinforces my view that BDB was stuck solely for wallet compatibility. I agree with the rationale of the second answer but why'd it take so long? BDB was a known pain for a long time. 

I guess in a nutshell I want to know if RocksDB/ArangoDB be used for both blockchain and wallet storage because moving away from BDB seemed long due.
Kittywhiskers Van Gogh
CC BY-SA 4.0
2021-02-12T16:57:14.333Z
Backwards compatibility is a massive priority for Core and it is generally very conservative in its technology choices. I don't think it would consider a new (database) technology until it had been proven over many years on other projects. It is providing a consensus engine and infrastructure and doesn't have the same tolerance as startups or bigger corporations for cycling through different options and experimentation if it is going to endanger backwards compatibility or require many developer hours to revamp shortly afterwards.
Michael Folkson
CC BY-SA 4.0
2021-02-12T17:02:11.930Z
That's what it seemed like, I know the maintainers went so far as to create a PPA just to have libdb4.8-dev as a package on Debian-based systems and can't imagine any other reason as to why they'd continue with BDB. I was wondering if there were technical advantages of BDB that I might've looked over or if there were non-support related reasons as to why BDB was stuck to for so long. Thanks for your help! <3
Kittywhiskers Van Gogh
CC BY-SA 4.0
2021-02-12T17:08:48.343Z
No problem. If that answered your question I'll add it as an answer.
Michael Folkson
<bitcoin-core><blockchain><bitcoin-core-development><database>
Is there any reason for an alternative Bitcoin implementation to use Berkeley DB?
Kittywhiskers Van Gogh
# Is there any reason for an alternative Bitcoin implementation to use Berkeley DB?

(Asked by: username_0 on 2021-02-12)

The primary reason Bitcoin's C++ reference implementation uses Berkeley DB 4.8 seems to be [backwards compatibility](https://github.com/bitcoin/bitcoin/issues/3686#issuecomment-74836708). It doesn't appear that BDB4.8 (or BDB at all for that matter) has any superior advantage over current-day alternatives like Redis or RocksDB, at least based on the limited research I've done ([source](https://db-engines.com/en/system/LevelDB%3BOracle+Berkeley+DB%3BRocksDB)). To the best of my knowledge, most alternative full-node implementations steer away from wallet support ([source](https://github.com/rust-bitcoin/rust-bitcoin), [source](https://github.com/MetacoSA/NBitcoin)).

Performance benchmarks seem to indicate that there is no performance benefit to sticking to Berkeley DB ([source](http://highscalability.com/blog/2012/11/29/performance-data-for-leveldb-berkley-db-and-bangdb-for-rando.html)) yet I cannot seem to see any widespread adoption of anything else except for recent introduction of SQLite support ([source](https://github.com/bitcoin/bitcoin/pull/19077)) for Bitcoin Core despite the growing pains of BDB4.8. I can't imagine any other reason as to why BDB wasn't dumped altogether if it wasn't for backwards support.

Is there any reason why alternatives like RocksDB (which supports [at rest encryption](https://github.com/facebook/rocksdb/blob/master/env/env_encryption.cc) though it isn't well documented), ArangoDB or Aerospike (all that seem to have existed at least five years ago) were not considered? Is there any security benefit to sticking with BDB? Any redeeming properties of BDB that make it a *definitive choice* over what is available currently?

Should a new implementation stick to BDB or use another library for wallet support and if so, what should it be?

**Tags:** bitcoin-core, blockchain, bitcoin-core-development, database

> **Comments:**
> **username_1**: There are a couple of questions here that at least partly answer your question (not the historical decision part though) At least if your needs are similar to the Core wallet today you would probably choose to use sqlite for the wallet rather than BDB https://bitcoin.stackexchange.com/questions/51435/migration-from-berkeley-db-to-leveldb?rq=1 https://bitcoin.stackexchange.com/questions/99620/why-is-the-bitcoin-core-wallet-database-moving-from-berkeley-db-to-sqlite  
> **username_0**: I'm familiar with both answers, the first one explains the transition away from BDB for blockchain storage and simply reinforces my view that BDB was stuck solely for wallet compatibility. I agree with the rationale of the second answer but why'd it take so long? BDB was a known pain for a long time.   I guess in a nutshell I want to know if RocksDB/ArangoDB be used for both blockchain and wallet storage because moving away from BDB seemed long due.  
> **username_1**: Backwards compatibility is a massive priority for Core and it is generally very conservative in its technology choices. I don't think it would consider a new (database) technology until it had been proven over many years on other projects. It is providing a consensus engine and infrastructure and doesn't have the same tolerance as startups or bigger corporations for cycling through different options and experimentation if it is going to endanger backwards compatibility or require many developer hours to revamp shortly afterwards.  
> **username_0**: That's what it seemed like, I know the maintainers went so far as to create a PPA just to have libdb4.8-dev as a package on Debian-based systems and can't imagine any other reason as to why they'd continue with BDB. I was wondering if there were technical advantages of BDB that I might've looked over or if there were non-support related reasons as to why BDB was stuck to for so long. Thanks for your help! <3  
> **username_1**: No problem. If that answered your question I'll add it as an answer.  

---

## 2 answers

### Answer by username_1 on 2021-02-12. Score: 2 | Accepted answer

Backwards compatibility is a massive priority for Core and it is generally very conservative in its technology choices. I don't think it would consider a new (database) technology until it had been proven over many years on other projects. It is providing a consensus engine and infrastructure and doesn't have the same tolerance as startups or bigger corporations for cycling through different options and experimentation if it is going to endanger backwards compatibility or require many developer hours to revamp shortly afterwards.


---

### Answer by username_2 on 2021-02-12. Score: 1

> I can't imagine any other reason as to why BDB wasn't dumped altogether if it wasn't for backwards support.

There simply is none. The wallet file doesn't need to be a database, it could be a flat text file and that would operate just as well. BDB will remain for compatibility for the foreseeable future, and change is risky, so it remains.

> **Comments:**
> **username_3**: Perhaps worth pointing out that BDB is only used for wallet files, which tend to have extremely long lifetimes (just search this site for people trying to get their funds out of wallet.dat files that are a decade old). For other purposes LevelDB is used, and incompatible changes to those databases have been made several times, for performance or other reasons. It's only for the wallet file that it really doesn't matter what format is used (it's loaded entirely into memory, at least for now), and compatibility is paramount.  

---

