<p>You can download the historical data of coinmarketcap.com with my developed crawler: <a href="https://github.com/roNn23/coinmarketcap-historical-data-crawler" rel="nofollow noreferrer">https://github.com/roNn23/coinmarketcap-historical-data-crawler</a>. But it's only getting the <a href="https://coinmarketcap.com/historical/" rel="nofollow noreferrer">snapshots of coinmarketcap.com</a>, maybe it's to wide for your needs.</p>

CC BY-SA 3.0
2018-02-10T05:43:32.540Z
2018-02-10T05:43:32.540Z
roNn23
<p>You can use my website <a href="http://www.cryptodatasets.com" rel="nofollow noreferrer">www.cryptodatasets.com</a> which provides exactly what you need.
tick by tick historical BTC, ETH, and LTC prices and volume from Bitfinex and Hitbtc.
It's exactly what I use to backtest my strategies/bots.</p>

CC BY-SA 3.0
2017-11-23T09:23:15.613Z
neat! Do you plan on extending the available exchanges?
deepbrook
CC BY-SA 3.0
2017-11-24T15:40:46.380Z
Absolutely, working on it in my free time!
MegaHotel
CC BY-SA 3.0
2017-11-22T21:22:41.563Z
2017-11-22T21:22:41.563Z
MegaHotel
<p>Just put up a website <a href="https://www.cryptoarchive.com.au" rel="nofollow noreferrer">cryptoarchive.com.au</a> with the data sets I use for my own modeling. One minute OHLVC data is available for free, tick-level data - very reasonably priced. Data from Binance for now.</p>

CC BY-SA 4.0
2019-01-24T00:42:29.757Z
2019-01-24T00:42:29.757Z
aleck
<p>For historical data about crypto exchange rates and volumes you can check: </p>

<ul>
<li><a href="https://www.cryptoarchive.com.au/" rel="nofollow noreferrer">cryptoarchive</a></li>
<li><a href="https://coinmarketcap.com/" rel="nofollow noreferrer">coinmarketcap</a> </li>
</ul>

CC BY-SA 4.0
2019-04-05T10:28:06.147Z
This answer adds no new information, unfortunately.
deepbrook
CC BY-SA 4.0
2019-04-05T10:00:18.137Z
2019-04-05T10:00:18.137Z
Justyna Laskowska
<p>If you're especially looking at quality of data, Coinscious has the most <a href="https://medium.com/coinscious/accurate-crypto-market-data-ultimately-leads-to-winning-model-13e7e421e546" rel="nofollow noreferrer">accurate</a> data. They offer both historical and real-time market data through flat files or API access. Their <a href="https://terminal.coinscious.io" rel="nofollow noreferrer">terminal</a> also has good technical analytics on different coins, exchanges and indicators. </p>

CC BY-SA 4.0
2019-06-06T18:08:10.973Z
2019-06-06T18:08:10.973Z
Johnson A
<h1>Acquiring Data</h1>

<p>You have two options here:</p>

<ol>
<li><strong>Check out one of the data providers online</strong>. <a href="https://www.quandl.com/search?query=bitcoin" rel="nofollow noreferrer">Quandl</a> offers (mostly) free historical data for a variety of pairs and exchanges. Alternatively, if you have bucks to spare, <a href="https://www.coinigy.com/bitcoin-data/" rel="nofollow noreferrer">Coinigy</a> offers high-quality data sets on a per-month pricing model.</li>
<li><strong>Roll your own data crawler</strong>, using a programming language of your choice and the various APIs available for exchanges.</li>
</ol>

<p>I'll discuss the two options further below.</p>

<hr>

<h1><strong>Acquiring Data from a Data Provider</strong></h1>

<p>Obviously, this option is the most convenient, however also possibly most expensive, depending on your required quality of the data. 
there is also no guarantee that the exchange, pair and/or time range you're looking for is present in the data base.</p>

<h2>To Sum up:</h2>

<p><strong>Pro:</strong></p>

<ul>
<li>Least time-consuming</li>
<li>Data is likely to be cleaned and formatted</li>
<li>Many data providers offer a unified layout of data across several data sets, making it easy to compare them.</li>
</ul>

<p><strong>Cons:</strong></p>

<ul>
<li>Paid data is expensive for hobbyists (Market data at coinigy is at 30$ / Month, as of the time of writing)</li>
<li>Free data is often provided as is, leaving you with clean-up duty and post-processing tasks</li>
<li>If data is cleaned, artifacts can occur without you knowing, tempering with your research results.</li>
</ul>

<hr>

<h1>Rolling your own Data Crawler</h1>

<p>With just a little experience in programming and basic understanding of how an API works, you can quickly set up your own data acquisition tool. There's varying degrees of difficulty, of course, but at the very heart of it, the matter is quite simple:</p>

<ol>
<li>Choose an Exchange</li>
<li>Choose an API</li>
<li>Hook into API using code wizardry</li>
<li>Start downloading data</li>
</ol>

<p>I'll walk you through the necessary steps (sans the code - I feel like this is a topic for another StackExchange; I will, however, link you to libraries that will help you get started).</p>

<h2>1. Choose an Exchange</h2>

<p>I'm guessing you've done your research and thusly, will only point out a few things when considering the exchanges you want to fetch data for:</p>

<ul>
<li><strong>Make sure they have stable servers and connections</strong> (Exchanges in Asia, accessed from, for example, Europe, have notoriously unstable connections). Otherwise you'll have pot holes all over your data, perhaps even rendering it useless.

<ul>
<li>if you absolutely require the data, consider renting a server closer to the exchange server center's physical location.</li>
</ul></li>
<li><strong>Make sure they have meaningful volume</strong>. If there's only little volume changing owners over the course of a set time frame (usually, the 24h volume metric is always available), the exchange is likely not a very good choice. It skew the picture of the market.</li>
<li><strong>Test their customer support first</strong>. If you happen to require their assistance some time later during crawling, a great support staff can make all the difference. Besides, it gives you a small indication of the exchange infrastructure's quality.</li>
</ul>

<h2>2. Choose an <code>API</code></h2>

<p>Choosing the right Application Programming Interface (<code>API</code>) depends on 2 things: </p>

<ol>
<li>How granular you need your data to be, </li>
</ol>

<p>and </p>

<ol start="2">
<li>what <code>API</code>s the exchange has to offer (obviously).</li>
</ol>

<h3><code>REST API</code>s</h3>

<p>If you're fine with, say, 1-Minute snapshots of data (i.e. tickers, order_books, trades etc), requesting data via a <a href="https://en.wikipedia.org/wiki/Representational_state_transfer" rel="nofollow noreferrer">Representational State Transfer</a> (REST) API is sufficient. This allows you to send a <code>http</code> request to a specified URL, and receive a response containing the data requested (usually in <code>JSON</code>-format). </p>

<p>The convenient thing about <code>REST</code> APIs is that they function mostly the same across exchanges - requests are sent, responses received and data evaluated by you. Some exchanges require you to use different request methods (such as <code>POST</code>) instead of the usual <code>GET</code> requests; however, this is usually only required for private endpoints (private meaning that you need to authenticate yourself first, before receiving data like your account's balance, etc), so if you're just after market data you shouldn't encounter them often. </p>

<p>A limitation of <code>REST</code> APIs is that they usually feature a request limit. The most commonly seen limit is <code>60 requests per minute</code>, but can be more strict (or more loose - Bitfinex allows unlimited <code>requests</code> per minute). Some exchanges also employ a request counter - this means that certain requests increase your <code>IP</code>-connected counter by a certain value. Depending on your status with the exchange this counter decreases over time by a pre-determined value. </p>

<p>In addition to this (if you're thinking 'Well, hey! Why not fetch data every second then?'), market data is often cached. A snapshot of an API endpoint is stored for a set interval, before actually being updated on the server side. Hence, you might send a request every second, but nonetheless receive identical data until the server cache is refreshed.</p>

<h3><code>WS API</code>s</h3>

<p><a href="https://en.wikipedia.org/wiki/WebSocket" rel="nofollow noreferrer">WebSocket</a> (<code>WS</code>) and WebSocketSecure (<code>WSS</code>) APIs are full-duplex connections, which allow the user to receive close-to-real-time updates of one or several <code>API</code> endpoints to which they've subscribed.
Full-duplex simply means that you can not only receive, but also send data to the <code>WebSocket</code> connection. This is predominantly useful when running a bot that is supposed to trade on your behalf, as the feedback is usually faster.</p>

<p>In principle, you open a connection to the <code>WS</code> API, and subscribe to the endpoints you want data on. Typically, this is separated by endpoint and pair, but make sure to consult the <code>API</code> documentation - some exchanges do not employ a <code>channel subscription</code> model, and data comes flooding right in without further configuration.</p>

<p>The implementation of <code>WebSocket</code>s ranges from trivial to complex, so expect to do some reasearch on how to access some of the exchanges' <code>WS</code> APIs. While the principle system remains identical across all of them, many exchanges use different protocols and/or services to provide their data - for example the <code>WAMP</code> protocol and Pusher, which add an additional layer of complexity, since they require a specific client to connect. </p>

<p>However, if you require tick-by-tick data, such as for real-time order book building, this is what you want to go with.</p>

<h3><code>FIX API</code>s</h3>

<p>The <code>Financial Information eXchange</code> (<code>FIX</code>) Protocol <a href="https://en.wikipedia.org/wiki/Financial_Information_eXchange" rel="nofollow noreferrer">is a standard started in 1992</a>, and now commonly used by institutions and brokers in the financial markets. It is by far the least available <code>API</code> at exchanges, with even fewer actually sending market data. It is most commonly used to place or cancel orders (for example at <a href="https://docs.pro.coinbase.com/#fix-api" rel="nofollow noreferrer">CoinbasePro</a>). However, some exchanges offer real-time market data via FIX. </p>

<p>Keep in mind that you'll need extensive knowledge of the FIX protocol, such as setting up a FIX engine and proxies to connect to the exchange. This may, <a href="https://docs.gemini.com/fix-api/#types-of-connectivity" rel="nofollow noreferrer">in some cases</a>, incur monthly-recurring charges.</p>

<h2>3. Hook into API using Code</h2>

<p>Depending on what API you chose, as well as which programming language you intend to use, you have a wide variety of options. </p>

<p>Many exchanges provide client libraries for their APIs, some developed by the exchange dev team themselves, others are contributed by users (which are usually revised by the developer team before being quoted on the exchange's website). </p>

<p>There are many other libraries out there, for likely any language (for example <a href="https://github.com/nlsdfnbch/bitex" rel="nofollow noreferrer">BitEx</a> for Python 3.x (to indulge in some shame-less self-promotion), which unifies core methods across a variety of <code>REST</code> <code>API</code>s, as well as some <code>WebSocket</code> <code>API</code>s).</p>

<p>Being a Python developer myself, I can only recommend using it. Especially if you're not bound to an environment. 
Using the <code>requests</code> library, for example, you can query an exchange with ease:</p>

<pre><code>import requests

# Get a list of all asset pairs at Kraken.com 
print(requests.get('https://api.kraken.com/0/public/AssetPairs').json())
</code></pre>

<h2>4.Start downloading data</h2>

<p>Once you've set up your data crawlers, you need to release them! I personally run the REST crawler via <code>cron</code> jobs, and <code>Websocket</code> crawlers as <code>daemons</code>. But this is up to you.</p>

<p>It will of course take some time to acquire a meaningful set of data, but alas, you either pay with money or with time for the data you want.</p>

<h1>Useful Links</h1>

<p><strong>API References</strong></p>

<ul>
<li><a href="https://en.wikipedia.org/wiki/Representational_state_transfer" rel="nofollow noreferrer">REST</a></li>
<li><a href="https://en.wikipedia.org/wiki/WebSocket" rel="nofollow noreferrer">WebSockets</a></li>
<li><a href="https://www.pubnub.com/blog/2015-01-05-websockets-vs-rest-api-understanding-the-difference/" rel="nofollow noreferrer">REST vs. WebSocket</a></li>
<li><a href="https://en.wikipedia.org/wiki/Financial_Information_eXchange" rel="nofollow noreferrer">FIX Protocol</a></li>
</ul>

<p><strong>Data Sources</strong></p>

<ul>
<li><a href="https://www.quandl.com/search?query=bitcoin" rel="nofollow noreferrer">Quandl.com</a></li>
<li><a href="https://www.coinigy.com/bitcoin-data/" rel="nofollow noreferrer">Coinigy.com</a></li>
<li><a href="http://www.coindesk.com/price/" rel="nofollow noreferrer">CoinDesk.com</a></li>
<li><a href="https://www.investing.com/currencies/btc-usd-historical-data" rel="nofollow noreferrer">Investing.com</a></li>
<li><a href="https://bitcoinchain.com/markets/btcusd-okcoin" rel="nofollow noreferrer">Bitcoinchain.com</a></li>
<li><a href="http://www.cryptodatasets.com" rel="nofollow noreferrer">Cryptodatasets.com</a></li>
<li><a href="https://coinmarketcap.com/historical/" rel="nofollow noreferrer">CoinMarketCap.com</a></li>
<li><a href="https://www.cryptoarchive.com.au/" rel="nofollow noreferrer">Crypto Archive</a></li>
</ul>

<p>Feel free to add, correct or update any of this content in the comments below! Thanks.</p>

CC BY-SA 4.0
2018-06-25T13:08:37.240Z
upvoted! any suggestions on how to go about websocket clients in python if you wanted real time data from say one or more of these APIs. I am a newbie and quite overwhelmed with the options available 1) tornado 2) twisted asyncio 3) asyncio 4) socket_io client 5) gevent 6) eventlet 7) threading with event loops per thread, what would be the right way to say fetch streams from several sources for your flask app, i am already using celery, thanks for the answer in advance!
PirateApp
CC BY-SA 4.0
2018-06-25T13:56:44.423Z
It depends a bit on the API and what protocol it uses (pusher, pure wss, WAMP, etc) but generally I just went with what was easiest to get started with. I've used the websocket-client library most of the time, as that gets the job done well enough for me. Each of the clients run in a separate process and I pull everything together using ZMQ. As to what you should go with: roll a dice ? :D they're all good choices.
deepbrook
CC BY-SA 4.0
2018-06-25T14:00:52.687Z
thanks for the detailed info :) does the javascript socket io client abstract the underlying details of wamp pusher and pure wss because i swear today is the first time i am hearing those especially when trying to do stuff with python, if it isnt too much to ask, is there a reference to this code online that you are speaking about which uses multiple processes and zmq, hours on github searching under the socketio tag or the websocket tag hasnt yielded anything positive, there are a few socketio clients on python as far as i researched and am not sure if they are battle tested, thanks
PirateApp
CC BY-SA 4.0
2018-06-26T07:36:20.987Z
As I haven't used socket.io, I don't know - but in principle, WSS and pusher are just protocols - they still run via websockets, but require a specific procedure when interacting with the API. https://github.com/Crypto-toolbox/Thoth has a pusher and WSS client and using zmq to offer the data. Usually there's a few libraries linked on the respective exchange's api doc site which offer working clients, though.
deepbrook
CC BY-SA 4.0
2018-06-26T09:07:41.543Z
thank you for sharing the link! as of now i am considering fetching the data in node.js since the socket io client on node is official and much more battle tested than the python ones, i might be wrong though, my idea was to redis pub sub the data but out of plain curiosity, how do you accumulate the data from streams? i am assuming disk writes would mean a lot of latency , redis would take say 200 ms itself and how do you tackle the case where say the socket disconnected for a few mins under heavy load
PirateApp
CC BY-SA 4.0
2018-06-27T07:01:23.190Z
I connect to the WSS and use ZMQ with a Pub/Sub pattern, where subscribers have a load balancer (several workers to handle incoming streams). I do write data to disk, but I do so at an interval defined by 400Kib cached or 5minutes passed since last write, whatever comes to pass first. I don't care about disconnects - If data is missing, there is nothing I can do about it anyway, although so far I haven't had the case that my socket disconnected due to heavy load, since all it does is relay it.
deepbrook
CC BY-SA 4.0
2018-06-27T07:25:10.527Z
wish i could up-vote these comments more than once! when you say workers i am assuming celery type workers that are capable of self restarting etc, which language do you personally prefer for fetching the socket data (node, python) Python seems to be more than capable of processing the data with pandas and what not but seems a bit limited when it comes to accepting socket client streams, maybe i am wrong
PirateApp
CC BY-SA 4.0
2018-06-27T07:42:13.230Z
Yes, you can compare them to celery workers. And I do use python - ultimately though, it doesn't really matter. ZMQ is pretty much language agnostic and has wrappers for all kinds of languages - I stuck with python mostly because I work with it daily.
deepbrook
CC BY-SA 4.0
2017-06-20T13:35:03.840Z
2019-04-05T10:28:45.390Z
2019-04-05T10:28:45.390Z
deepbrook
<p>I'm looking for sources of crypto currency data, preferably historical, which I can use to back-test my bots and/or analyze the market.</p>

<p>Is there a reliable source for this online ? Do exchanges offer historical data by any chance?</p>

<p>If not, how do I go about acquiring this data?</p>

<currencies><raw-data><historical-trade-data><live-data>
Where can I find Crypto Coin data?
deepbrook
# Where can I find Crypto Coin data?

(Asked by: username_0 on 2017-06-20)

I'm looking for sources of crypto currency data, preferably historical, which I can use to back-test my bots and/or analyze the market.

Is there a reliable source for this online ? Do exchanges offer historical data by any chance?

If not, how do I go about acquiring this data?

**Tags:** currencies, raw-data, historical-trade-data, live-data


---

## 6 answers

### Answer by username_0 on 2017-06-20. Score: 10 | Accepted answer

# Acquiring Data

You have two options here:

1. **Check out one of the data providers online**. [Quandl](https://www.quandl.com/search?query=bitcoin) offers (mostly) free historical data for a variety of pairs and exchanges. Alternatively, if you have bucks to spare, [Coinigy](https://www.coinigy.com/bitcoin-data/) offers high-quality data sets on a per-month pricing model.
2. **Roll your own data crawler**, using a programming language of your choice and the various APIs available for exchanges.

I'll discuss the two options further below.

---

# **Acquiring Data from a Data Provider**

Obviously, this option is the most convenient, however also possibly most expensive, depending on your required quality of the data.
there is also no guarantee that the exchange, pair and/or time range you're looking for is present in the data base.

## To Sum up:

**Pro:**

* Least time-consuming
* Data is likely to be cleaned and formatted
* Many data providers offer a unified layout of data across several data sets, making it easy to compare them.

**Cons:**

* Paid data is expensive for hobbyists (Market data at coinigy is at 30$ / Month, as of the time of writing)
* Free data is often provided as is, leaving you with clean-up duty and post-processing tasks
* If data is cleaned, artifacts can occur without you knowing, tempering with your research results.

---

# Rolling your own Data Crawler

With just a little experience in programming and basic understanding of how an API works, you can quickly set up your own data acquisition tool. There's varying degrees of difficulty, of course, but at the very heart of it, the matter is quite simple:

1. Choose an Exchange
2. Choose an API
3. Hook into API using code wizardry
4. Start downloading data

I'll walk you through the necessary steps (sans the code - I feel like this is a topic for another StackExchange; I will, however, link you to libraries that will help you get started).

## 1. Choose an Exchange

I'm guessing you've done your research and thusly, will only point out a few things when considering the exchanges you want to fetch data for:

* **Make sure they have stable servers and connections** (Exchanges in Asia, accessed from, for example, Europe, have notoriously unstable connections). Otherwise you'll have pot holes all over your data, perhaps even rendering it useless.
  + if you absolutely require the data, consider renting a server closer to the exchange server center's physical location.
* **Make sure they have meaningful volume**. If there's only little volume changing owners over the course of a set time frame (usually, the 24h volume metric is always available), the exchange is likely not a very good choice. It skew the picture of the market.
* **Test their customer support first**. If you happen to require their assistance some time later during crawling, a great support staff can make all the difference. Besides, it gives you a small indication of the exchange infrastructure's quality.

## 2. Choose an `API`

Choosing the right Application Programming Interface (`API`) depends on 2 things:

1. How granular you need your data to be,

and

2. what `API`s the exchange has to offer (obviously).

### `REST API`s

If you're fine with, say, 1-Minute snapshots of data (i.e. tickers, order\_books, trades etc), requesting data via a [Representational State Transfer](https://en.wikipedia.org/wiki/Representational_state_transfer) (REST) API is sufficient. This allows you to send a `http` request to a specified URL, and receive a response containing the data requested (usually in `JSON`-format).

The convenient thing about `REST` APIs is that they function mostly the same across exchanges - requests are sent, responses received and data evaluated by you. Some exchanges require you to use different request methods (such as `POST`) instead of the usual `GET` requests; however, this is usually only required for private endpoints (private meaning that you need to authenticate yourself first, before receiving data like your account's balance, etc), so if you're just after market data you shouldn't encounter them often.

A limitation of `REST` APIs is that they usually feature a request limit. The most commonly seen limit is `60 requests per minute`, but can be more strict (or more loose - Bitfinex allows unlimited `requests` per minute). Some exchanges also employ a request counter - this means that certain requests increase your `IP`-connected counter by a certain value. Depending on your status with the exchange this counter decreases over time by a pre-determined value.

In addition to this (if you're thinking 'Well, hey! Why not fetch data every second then?'), market data is often cached. A snapshot of an API endpoint is stored for a set interval, before actually being updated on the server side. Hence, you might send a request every second, but nonetheless receive identical data until the server cache is refreshed.

### `WS API`s

[WebSocket](https://en.wikipedia.org/wiki/WebSocket) (`WS`) and WebSocketSecure (`WSS`) APIs are full-duplex connections, which allow the user to receive close-to-real-time updates of one or several `API` endpoints to which they've subscribed.
Full-duplex simply means that you can not only receive, but also send data to the `WebSocket` connection. This is predominantly useful when running a bot that is supposed to trade on your behalf, as the feedback is usually faster.

In principle, you open a connection to the `WS` API, and subscribe to the endpoints you want data on. Typically, this is separated by endpoint and pair, but make sure to consult the `API` documentation - some exchanges do not employ a `channel subscription` model, and data comes flooding right in without further configuration.

The implementation of `WebSocket`s ranges from trivial to complex, so expect to do some reasearch on how to access some of the exchanges' `WS` APIs. While the principle system remains identical across all of them, many exchanges use different protocols and/or services to provide their data - for example the `WAMP` protocol and Pusher, which add an additional layer of complexity, since they require a specific client to connect.

However, if you require tick-by-tick data, such as for real-time order book building, this is what you want to go with.

### `FIX API`s

The `Financial Information eXchange` (`FIX`) Protocol [is a standard started in 1992](https://en.wikipedia.org/wiki/Financial_Information_eXchange), and now commonly used by institutions and brokers in the financial markets. It is by far the least available `API` at exchanges, with even fewer actually sending market data. It is most commonly used to place or cancel orders (for example at [CoinbasePro](https://docs.pro.coinbase.com/#fix-api)). However, some exchanges offer real-time market data via FIX.

Keep in mind that you'll need extensive knowledge of the FIX protocol, such as setting up a FIX engine and proxies to connect to the exchange. This may, [in some cases](https://docs.gemini.com/fix-api/#types-of-connectivity), incur monthly-recurring charges.

## 3. Hook into API using Code

Depending on what API you chose, as well as which programming language you intend to use, you have a wide variety of options.

Many exchanges provide client libraries for their APIs, some developed by the exchange dev team themselves, others are contributed by users (which are usually revised by the developer team before being quoted on the exchange's website).

There are many other libraries out there, for likely any language (for example [BitEx](https://github.com/nlsdfnbch/bitex) for Python 3.x (to indulge in some shame-less self-promotion), which unifies core methods across a variety of `REST` `API`s, as well as some `WebSocket` `API`s).

Being a Python developer myself, I can only recommend using it. Especially if you're not bound to an environment.
Using the `requests` library, for example, you can query an exchange with ease:

```
import requests

# Get a list of all asset pairs at Kraken.com 
print(requests.get('https://api.kraken.com/0/public/AssetPairs').json())

```

## 4.Start downloading data

Once you've set up your data crawlers, you need to release them! I personally run the REST crawler via `cron` jobs, and `Websocket` crawlers as `daemons`. But this is up to you.

It will of course take some time to acquire a meaningful set of data, but alas, you either pay with money or with time for the data you want.

# Useful Links

**API References**

* [REST](https://en.wikipedia.org/wiki/Representational_state_transfer)
* [WebSockets](https://en.wikipedia.org/wiki/WebSocket)
* [REST vs. WebSocket](https://www.pubnub.com/blog/2015-01-05-websockets-vs-rest-api-understanding-the-difference/)
* [FIX Protocol](https://en.wikipedia.org/wiki/Financial_Information_eXchange)

**Data Sources**

* [Quandl.com](https://www.quandl.com/search?query=bitcoin)
* [Coinigy.com](https://www.coinigy.com/bitcoin-data/)
* [CoinDesk.com](http://www.coindesk.com/price/)
* [Investing.com](https://www.investing.com/currencies/btc-usd-historical-data)
* [Bitcoinchain.com](https://bitcoinchain.com/markets/btcusd-okcoin)
* [Cryptodatasets.com](http://www.cryptodatasets.com)
* [CoinMarketCap.com](https://coinmarketcap.com/historical/)
* [Crypto Archive](https://www.cryptoarchive.com.au/)

Feel free to add, correct or update any of this content in the comments below! Thanks.

> **Comments:**
> **username_6**: upvoted! any suggestions on how to go about websocket clients in python if you wanted real time data from say one or more of these APIs. I am a newbie and quite overwhelmed with the options available 1) tornado 2) twisted asyncio 3) asyncio 4) socket_io client 5) gevent 6) eventlet 7) threading with event loops per thread, what would be the right way to say fetch streams from several sources for your flask app, i am already using celery, thanks for the answer in advance!  
> **username_0**: It depends a bit on the API and what protocol it uses (pusher, pure wss, WAMP, etc) but generally I just went with what was easiest to get started with. I've used the websocket-client library most of the time, as that gets the job done well enough for me. Each of the clients run in a separate process and I pull everything together using ZMQ. As to what you should go with: roll a dice ? :D they're all good choices.  
> **username_6**: thanks for the detailed info :) does the javascript socket io client abstract the underlying details of wamp pusher and pure wss because i swear today is the first time i am hearing those especially when trying to do stuff with python, if it isnt too much to ask, is there a reference to this code online that you are speaking about which uses multiple processes and zmq, hours on github searching under the socketio tag or the websocket tag hasnt yielded anything positive, there are a few socketio clients on python as far as i researched and am not sure if they are battle tested, thanks  
> **username_0**: As I haven't used socket.io, I don't know - but in principle, WSS and pusher are just protocols - they still run via websockets, but require a specific procedure when interacting with the API. https://github.com/Crypto-toolbox/Thoth has a pusher and WSS client and using zmq to offer the data. Usually there's a few libraries linked on the respective exchange's api doc site which offer working clients, though.  
> **username_6**: thank you for sharing the link! as of now i am considering fetching the data in node.js since the socket io client on node is official and much more battle tested than the python ones, i might be wrong though, my idea was to redis pub sub the data but out of plain curiosity, how do you accumulate the data from streams? i am assuming disk writes would mean a lot of latency , redis would take say 200 ms itself and how do you tackle the case where say the socket disconnected for a few mins under heavy load  
> **username_0**: I connect to the WSS and use ZMQ with a Pub/Sub pattern, where subscribers have a load balancer (several workers to handle incoming streams). I do write data to disk, but I do so at an interval defined by 400Kib cached or 5minutes passed since last write, whatever comes to pass first. I don't care about disconnects - If data is missing, there is nothing I can do about it anyway, although so far I haven't had the case that my socket disconnected due to heavy load, since all it does is relay it.  
> **username_6**: wish i could up-vote these comments more than once! when you say workers i am assuming celery type workers that are capable of self restarting etc, which language do you personally prefer for fetching the socket data (node, python) Python seems to be more than capable of processing the data with pandas and what not but seems a bit limited when it comes to accepting socket client streams, maybe i am wrong  
> **username_0**: Yes, you can compare them to celery workers. And I do use python - ultimately though, it doesn't really matter. ZMQ is pretty much language agnostic and has wrappers for all kinds of languages - I stuck with python mostly because I work with it daily.  

---

### Answer by username_1 on 2018-02-10. Score: 2

You can download the historical data of coinmarketcap.com with my developed crawler: <https://github.com/username_1/coinmarketcap-historical-data-crawler>. But it's only getting the [snapshots of coinmarketcap.com](https://coinmarketcap.com/historical/), maybe it's to wide for your needs.


---

### Answer by username_2 on 2017-11-22. Score: 1

You can use my website [www.cryptodatasets.com](http://www.cryptodatasets.com) which provides exactly what you need.
tick by tick historical BTC, ETH, and LTC prices and volume from Bitfinex and Hitbtc.
It's exactly what I use to backtest my strategies/bots.

> **Comments:**
> **username_0**: neat! Do you plan on extending the available exchanges?  
> **username_2**: Absolutely, working on it in my free time!  

---

### Answer by username_3 on 2019-01-24. Score: 1

Just put up a website [cryptoarchive.com.au](https://www.cryptoarchive.com.au) with the data sets I use for my own modeling. One minute OHLVC data is available for free, tick-level data - very reasonably priced. Data from Binance for now.


---

### Answer by username_4 on 2019-04-05. Score: 0

For historical data about crypto exchange rates and volumes you can check:

* [cryptoarchive](https://www.cryptoarchive.com.au/)
* [coinmarketcap](https://coinmarketcap.com/)

> **Comments:**
> **username_0**: This answer adds no new information, unfortunately.  

---

### Answer by username_5 on 2019-06-06. Score: -1

If you're especially looking at quality of data, Coinscious has the most [accurate](https://medium.com/coinscious/accurate-crypto-market-data-ultimately-leads-to-winning-model-13e7e421e546) data. They offer both historical and real-time market data through flat files or API access. Their [terminal](https://terminal.coinscious.io) also has good technical analytics on different coins, exchanges and indicators.


---

