<p>As someone who was involved in doing that migration at the time, I believe it was the right decision. LevelDB is far from perfect, but I wouldn't know what else to use.</p>

<p>In particular:</p>

<ul>
<li>BDB is much slower for our usage (large atomic batch writes, small random reads).</li>
<li>There were reports of database corruption as well with BDB, at a time when it was used far less intensively than LevelDB is now.</li>
<li>BDB is very painful to upgrade. My impression is that it was designed for setups where a database upgrade only happened with professional supervision. In particular, the write log files created for durability were sometimes not readable by later versions. This is the reason why Bitcoin Core releases have for years stuck with BDB 4.8 for the wallet.</li>
<li>BDB has many resource limits that need configuration, where inappropriately chosen values may cause network-wide failures (read up on the <a href="https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki" rel="noreferrer">March 2013 incident</a>, specifically). The <a href="https://docs.oracle.com/cd/E17276_01/html/programmer_reference/lock_max.html" rel="noreferrer">official documentation</a> on this states <code>Reviewing the Lock subsystem statistics is the best way to determine this value.</code> I believe that's not acceptable for our use case.</li>
</ul>

<p>While database corruptions are reported relatively frequently these days, my belief is that it's mostly hardware failures or driver issues. Bitcoin Core tends to stress disks, memory and CPUs far more than most software, which makes otherwise invisible issues pop up. </p>

CC BY-SA 3.0
2017-02-17T06:19:18.707Z
_> "[...] database corruptions are reported relatively frequently these days [...]"_ Is there an active LevelDB community to provide some feedback/support about such issue?
Mark Messa
CC BY-SA 3.0
2017-02-17T06:46:55.437Z
I personally *never* see corruptions, but I run on relatively new hardware, and use Linux. But I probably resync the chain on average once I week for development purposes. That doesn't mean I don't believe such reports are real, but at least in some cases they turn out to be due to flaky hardware or storage systems - in which case it's hard to blame the database software.
Pieter Wuille
CC BY-SA 3.0
2018-01-28T07:51:02.893Z
What about Redis? https://db-engines.com/en/system/LevelDB%3BOracle+Berkeley+DB%3BRedis
Zakaria
CC BY-SA 3.0
2017-02-17T03:16:32.517Z
2017-02-17T03:31:41.820Z
2017-02-17T03:31:41.820Z
Pieter Wuille
<p>In 2013 a new bitcoin core was released and one of the supposed improvement was migrating from Berkeley DB to LevelDB. According to the release notes at <a href="https://bitcoin.org/en/release/v0.8.1" rel="noreferrer">bitcoin.org</a>:  </p>

<blockquote>
  <p>LevelDB, a fast, open-source, non-relational database from Google, is
  now used to store transaction and block indices. <strong>LevelDB works much
  better on machines with slow I/O and is faster in general</strong>.</p>
</blockquote>

<p>Similar statements were done by <a href="https://bitcoin.stackexchange.com/questions/4809/what-technology-will-supersede-the-berkleydb-format-what-are-the-current-scalab">bitcoin core developers</a>:  </p>

<blockquote>
  <p>LevelDB - an open-source rewrite of Google's own database system - was
  designed for efficiency and consistency on commodity hardware, and
  <strong>outperforms BDB by an order of magnitude</strong> in some settings. Preliminary
  <strong>tests with LevelDB show very good results</strong>.</p>
</blockquote>

<p>However, nowadays there are several comments through the internet critizing LevelDB. According to <a href="https://en.wikipedia.org/wiki/LevelDB" rel="noreferrer">LevelDB page at Wikipedia</a>:  </p>

<blockquote>
  <p>LevelDB is widely noted for being unreliable and databases it manages
  are prone to corruption. Academic studies of past versions of LevelDB
  have found that, under some file systems, the data stored in those
  versions of LevelDB might become inconsistent after a system crash or
  power failure. LevelDB corruption is so commonplace that corruption
  detection has to be built in to applications that use it.</p>
</blockquote>

<p>In summary, the <a href="https://stackoverflow.com/questions/18305031/whats-the-status-of-leveldb-is-it-safe-for-use-in-production">complaints about LevelDB</a> are:  </p>

<ul>
<li>unreliable and databases are prone to corruption</li>
<li>several bugs due to fundamental flaws</li>
<li>poor code quality</li>
<li>no longer actively maintained (i.e. dead project)</li>
</ul>

<p><strong>Question 1</strong><br>
Are these complaints about LevelDB correct?</p>

<p><strong>Question 2</strong><br>
What is the final balance of migrating Bitcoin Core from Berkeley DB to LevelDB? Is LevelDB working as expected? This migration was the right choice?</p>

CC BY-SA 3.0
2017-02-16T20:02:17.573Z
IIRC, Bitcoin uses BerkleyDB 4.8 for wallet files, and LevelDB for blockchain indexes. Informally, I can tell you that blockchain index corruption during crashes is very common, and I usually tell affected users to redownload the blockchain.
Nick ODell
CC BY-SA 3.0
2017-02-16T20:59:26.193Z
@NickODell Do you know whether such crashes you`ve mentioned also happened before the migration to LevelDB?
Mark Messa
CC BY-SA 3.0
2017-02-16T21:23:46.787Z
Sorry, I don't know.
Nick ODell
CC BY-SA 4.0
2021-02-28T05:46:48.383Z
@NickODell, should I backup my Bitcoin Core data directory (eg. chainstate/ and blocks/) so that I wouldn't have to redownload the blockchain from block 0, in the event of LevelDB corruption?
Donn Lee
<bitcoin-core><bitcoin-core-development><database><leveldb>
Migration from Berkeley DB to LevelDB
Mark Messa
# Migration from Berkeley DB to LevelDB

(Asked by: username_0 on 2017-02-16)

In 2013 a new bitcoin core was released and one of the supposed improvement was migrating from Berkeley DB to LevelDB. According to the release notes at [bitcoin.org](https://bitcoin.org/en/release/v0.8.1):

> LevelDB, a fast, open-source, non-relational database from Google, is
> now used to store transaction and block indices. **LevelDB works much
> better on machines with slow I/O and is faster in general**.

Similar statements were done by [bitcoin core developers](https://bitcoin.stackexchange.com/questions/4809/what-technology-will-supersede-the-berkleydb-format-what-are-the-current-scalab):

> LevelDB - an open-source rewrite of Google's own database system - was
> designed for efficiency and consistency on commodity hardware, and
> **outperforms BDB by an order of magnitude** in some settings. Preliminary
> **tests with LevelDB show very good results**.

However, nowadays there are several comments through the internet critizing LevelDB. According to [LevelDB page at Wikipedia](https://en.wikipedia.org/wiki/LevelDB):

> LevelDB is widely noted for being unreliable and databases it manages
> are prone to corruption. Academic studies of past versions of LevelDB
> have found that, under some file systems, the data stored in those
> versions of LevelDB might become inconsistent after a system crash or
> power failure. LevelDB corruption is so commonplace that corruption
> detection has to be built in to applications that use it.

In summary, the [complaints about LevelDB](https://stackoverflow.com/questions/18305031/whats-the-status-of-leveldb-is-it-safe-for-use-in-production) are:

* unreliable and databases are prone to corruption
* several bugs due to fundamental flaws
* poor code quality
* no longer actively maintained (i.e. dead project)

**Question 1**  
Are these complaints about LevelDB correct?

**Question 2**  
What is the final balance of migrating Bitcoin Core from Berkeley DB to LevelDB? Is LevelDB working as expected? This migration was the right choice?

**Tags:** bitcoin-core, bitcoin-core-development, database, leveldb

> **Comments:**
> **username_1**: IIRC, Bitcoin uses BerkleyDB 4.8 for wallet files, and LevelDB for blockchain indexes. Informally, I can tell you that blockchain index corruption during crashes is very common, and I usually tell affected users to redownload the blockchain.  
> **username_0**: @username_1 Do you know whether such crashes you`ve mentioned also happened before the migration to LevelDB?  
> **username_1**: Sorry, I don't know.  
> **username_2**: @username_1, should I backup my Bitcoin Core data directory (eg. chainstate/ and blocks/) so that I wouldn't have to redownload the blockchain from block 0, in the event of LevelDB corruption?  

---

## 1 answers

### Answer by username_3 on 2017-02-17. Score: 17 | Accepted answer

As someone who was involved in doing that migration at the time, I believe it was the right decision. LevelDB is far from perfect, but I wouldn't know what else to use.

In particular:

* BDB is much slower for our usage (large atomic batch writes, small random reads).
* There were reports of database corruption as well with BDB, at a time when it was used far less intensively than LevelDB is now.
* BDB is very painful to upgrade. My impression is that it was designed for setups where a database upgrade only happened with professional supervision. In particular, the write log files created for durability were sometimes not readable by later versions. This is the reason why Bitcoin Core releases have for years stuck with BDB 4.8 for the wallet.
* BDB has many resource limits that need configuration, where inappropriately chosen values may cause network-wide failures (read up on the [March 2013 incident](https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki), specifically). The [official documentation](https://docs.oracle.com/cd/E17276_01/html/programmer_reference/lock_max.html) on this states `Reviewing the Lock subsystem statistics is the best way to determine this value.` I believe that's not acceptable for our use case.

While database corruptions are reported relatively frequently these days, my belief is that it's mostly hardware failures or driver issues. Bitcoin Core tends to stress disks, memory and CPUs far more than most software, which makes otherwise invisible issues pop up.

> **Comments:**
> **username_0**: _> "[...] database corruptions are reported relatively frequently these days [...]"_ Is there an active LevelDB community to provide some feedback/support about such issue?  
> **username_3**: I personally *never* see corruptions, but I run on relatively new hardware, and use Linux. But I probably resync the chain on average once I week for development purposes. That doesn't mean I don't believe such reports are real, but at least in some cases they turn out to be due to flaky hardware or storage systems - in which case it's hard to blame the database software.  
> **username_4**: What about Redis? https://db-engines.com/en/system/LevelDB%3BOracle+Berkeley+DB%3BRedis  

---

