<p>I would like to use the <code>bitcoind</code> daemon but I don't want the high CPU footprint related to validating blocks every time new blocks come in. There is a better equipped computer that I can <code>rsync</code> with, which would in theory allow me to just download the files and run:</p>

<pre><code>bitcoind --daemon -connect=0.0.0.0 -checklevel=0
</code></pre>

<p>Note that I run <code>connect=0.0.0.0</code> because I don't want new blocks to arrive directly on this computer, and <code>checklevel=0</code> to prevent any block checks upon start. </p>

<p>I sync this with command:</p>

<p><code>rsync --progress --partial --recursive --size-only --verbose --delete -e "ssh -v" bitcoinserver:/home/jny/.bitcoin/blocks/ /home/jny/.bitcoin/blocks/  &amp;&amp; scp bitcoinserver:/home/jny/.bitcoin/blocks/index/CURRENT /home/jny/.bitcoin/blocks/index/</code></p>

<p>So I sync the <code>blocks</code> directory and download the <code>CURRENT</code> file in the <code>index</code> directory. Through trial and error I found this to work. However, there is one problem: I still have a high CPU usage because of the UpdateTip function that runs to process the blocks (see: <a href="https://bitcoin.stackexchange.com/questions/29691/why-could-updatetip-be-taking-so-long">Why could UpdateTip be taking so long?</a>). </p>

<p>However, if I do include the <code>chainstate</code> directory in my syncing, then I can not start bitcoin:</p>

<pre><code>2014-07-25 12:03:36 Opened LevelDB successfully
2014-07-25 12:03:36 Opening LevelDB in /home/jny/.bitcoin/chainstate
2014-07-25 12:03:39 Corruption: 207 missing files; e.g.: /home/jny/.bitcoin/chainstate/452003.ldb
2014-07-25 12:03:39 : Error opening block database.

Do you want to rebuild the block database now?
2014-07-25 12:03:39 Aborted block database rebuild. Exiting.
2014-07-25 12:03:39 Shutdown : In progress...
</code></pre>

<p>I imagine that the problem is that the files in chainstate are being changed as I download them. Stopping bitcoind on the server while rsyncing is not an option, unfortunately. </p>

<p>Another disadvantage is that files in the <code>chainstate</code> directory seem to change a lot, which makes rsync download many of them over and over again, even though it could have been only a few hours since the last sync. </p>

<p>Any idea how to set up a more streamlined rsync of my bitcoin database? </p>

CC BY-SA 3.0
2014-07-25T14:29:38.093Z
Before going to all this trouble, have you profiled to check that the "CPU footprint" is really unacceptable? Once you have caught up with the network initially, there's only one block every ten minutes on average, and on a modern computer it should take only a second or two of CPU time to verify. I'm having trouble thinking of a situation where that would pose a problem.
Nate Eldredge
CC BY-SA 3.0
2014-07-25T17:38:25.087Z
Thanks @NateEldredge. But yes, I did try using bitcoind the normal way. The problem is that I can't run it continuously. I can only open and sync it every 2-10 days, so the problem is that all the blocks come at once and it takes a whole lot of CPU power to catch up.
user
CC BY-SA 3.0
2014-07-25T22:15:53.260Z
I don't think rsync is going to solve your CPU footprint, because the CPU intensive part comes from verifying the downloaded blocks, not from the downloading of the block. You are probably better off using a thin client like Multibit instead of a full client.
k kurokawa
CC BY-SA 3.0
2014-07-26T06:43:40.097Z
But I should be able to download the verification as well, no? Ultimately, it's all nothing but files, right? I really prefer to use the bitcoind client...
user
<bitcoind><synchronization>
How can I rsync my bitcoin database?
user
# How can I rsync my bitcoin database?

(Asked by: username_0 on 2014-07-25)

I would like to use the `bitcoind` daemon but I don't want the high CPU footprint related to validating blocks every time new blocks come in. There is a better equipped computer that I can `rsync` with, which would in theory allow me to just download the files and run:

```
bitcoind --daemon -connect=0.0.0.0 -checklevel=0

```

Note that I run `connect=0.0.0.0` because I don't want new blocks to arrive directly on this computer, and `checklevel=0` to prevent any block checks upon start.

I sync this with command:

`rsync --progress --partial --recursive --size-only --verbose --delete -e "ssh -v" bitcoinserver:/home/jny/.bitcoin/blocks/ /home/jny/.bitcoin/blocks/ && scp bitcoinserver:/home/jny/.bitcoin/blocks/index/CURRENT /home/jny/.bitcoin/blocks/index/`

So I sync the `blocks` directory and download the `CURRENT` file in the `index` directory. Through trial and error I found this to work. However, there is one problem: I still have a high CPU usage because of the UpdateTip function that runs to process the blocks (see: [Why could UpdateTip be taking so long?](https://bitcoin.stackexchange.com/questions/29691/why-could-updatetip-be-taking-so-long)).

However, if I do include the `chainstate` directory in my syncing, then I can not start bitcoin:

```
2014-07-25 12:03:36 Opened LevelDB successfully
2014-07-25 12:03:36 Opening LevelDB in /home/jny/.bitcoin/chainstate
2014-07-25 12:03:39 Corruption: 207 missing files; e.g.: /home/jny/.bitcoin/chainstate/452003.ldb
2014-07-25 12:03:39 : Error opening block database.

Do you want to rebuild the block database now?
2014-07-25 12:03:39 Aborted block database rebuild. Exiting.
2014-07-25 12:03:39 Shutdown : In progress...

```

I imagine that the problem is that the files in chainstate are being changed as I download them. Stopping bitcoind on the server while rsyncing is not an option, unfortunately.

Another disadvantage is that files in the `chainstate` directory seem to change a lot, which makes rsync download many of them over and over again, even though it could have been only a few hours since the last sync.

Any idea how to set up a more streamlined rsync of my bitcoin database?

**Tags:** bitcoind, synchronization

> **Comments:**
> **username_1**: Before going to all this trouble, have you profiled to check that the "CPU footprint" is really unacceptable? Once you have caught up with the network initially, there's only one block every ten minutes on average, and on a modern computer it should take only a second or two of CPU time to verify. I'm having trouble thinking of a situation where that would pose a problem.  
> **username_0**: Thanks @username_1. But yes, I did try using bitcoind the normal way. The problem is that I can't run it continuously. I can only open and sync it every 2-10 days, so the problem is that all the blocks come at once and it takes a whole lot of CPU power to catch up.  
> **username_2**: I don't think rsync is going to solve your CPU footprint, because the CPU intensive part comes from verifying the downloaded blocks, not from the downloading of the block. You are probably better off using a thin client like Multibit instead of a full client.  
> **username_0**: But I should be able to download the verification as well, no? Ultimately, it's all nothing but files, right? I really prefer to use the bitcoind client...  

---

## 0 answers

