<p>As far as I know, Prometheus doesn't mind high-cardinality <strong>data</strong>. What Prometheus doesn't like is high-cardinality <strong>labels</strong>.</p>

<p>Let's start with Prometheus official documentation, it gives a good high-level explanation why:</p>

<blockquote>
  <p>CAUTION: Remember that every unique combination of key-value label
  pairs represents a new time series, which can dramatically increase
  the amount of data stored. Do not use labels to store dimensions with
  high cardinality (many different label values), such as user IDs,
  email addresses, or other unbounded sets of values.</p>
</blockquote>

<ul>
<li><a href="https://prometheus.io/docs/practices/naming/" rel="noreferrer">https://prometheus.io/docs/practices/naming/</a></li>
</ul>

<p>The important part is really that <strong>unique combinations of key-value</strong> creates <strong>new time series</strong> in Prometheus.</p>

<p>For instance, if you store a gauge-type metric <code>registration_complete</code> that records the time a user took to complete his/her registration form, Prometheus won't have any issue with that. You'll have one time series that have hundred of thousand of values: one value for each user that registered (how long it took). You'll be able to graph that metric over time, get the p95, etc.</p>

<p>If you want to add <em>some</em> cardinality to it, you could add a label like <code>region</code>: US-east, Asia-Pacific, etc. You'll be able to graph all regions and compare them or maybe group them all. The number of regions is probably low (&lt;10) and it is certainly bound. If on AWS, there's a fixed number of regions and they don't change much over time. Sure, AWS might add, remove or rename a region, but that's not changing by the minute and there's not thousands of them.</p>

<p>So to come back to what Prometheus advise against: you should not create high-cardinality labels. You should not add a <code>user_id</code> label to your <code>registration_complete</code> metric. If you do so, you'll have hundred of thousands of <em>different</em> time series, one for each user!, and they will all have only one data point. That's really the worst case scenario.</p>

<p>In this case, in order to graph the <code>registration_complete</code> metric over all labels, Prometheus will have to query all the individual time series (the X thousands of them) and aggregate all of them.</p>

<p>You said you come from a SQL background, so I'll attempt an analogy. Unique combination of key-value label pairs creating new time series in Prometheus amounts to individual tables in SQL. Having a label <code>user_id</code> equals to having one table per user_id.</p>

<p>Note: not all TSDB work the same way and I can't speak for them all.</p>

CC BY-SA 4.0
2019-05-22T14:52:27.083Z
2019-05-23T00:20:51.383Z
2019-05-23T00:20:51.383Z
Alexandre
<p>I have a background in relational databases and new to Prometheus. I wonder why Prometheus is not a good choice for high cardinality data? Why do I need to use low cardinality data? It's exact opposite from SQL DBs.</p>

<p>What are the technical reason for that?</p>

CC BY-SA 4.0
2019-05-23T21:08:28.770Z
This is a FAQ: https://prometheus.io/docs/practices/naming/
Martin Schröder
<monitoring><databases><prometheus><sql><data>
Why is Prometheus not a good choice for data with high cardinality?
Sybil
# Why is Prometheus not a good choice for data with high cardinality?

(Asked by: username_0 on 2019-05-21)

I have a background in relational databases and new to Prometheus. I wonder why Prometheus is not a good choice for high cardinality data? Why do I need to use low cardinality data? It's exact opposite from SQL DBs.

What are the technical reason for that?

**Tags:** monitoring, databases, prometheus, sql, data

> **Comments:**
> **username_1**: This is a FAQ: https://prometheus.io/docs/practices/naming/  

---

## 1 answers

### Answer by username_2 on 2019-05-22. Score: 13 | Accepted answer

As far as I know, Prometheus doesn't mind high-cardinality **data**. What Prometheus doesn't like is high-cardinality **labels**.

Let's start with Prometheus official documentation, it gives a good high-level explanation why:

> CAUTION: Remember that every unique combination of key-value label
> pairs represents a new time series, which can dramatically increase
> the amount of data stored. Do not use labels to store dimensions with
> high cardinality (many different label values), such as user IDs,
> email addresses, or other unbounded sets of values.

* <https://prometheus.io/docs/practices/naming/>

The important part is really that **unique combinations of key-value** creates **new time series** in Prometheus.

For instance, if you store a gauge-type metric `registration_complete` that records the time a user took to complete his/her registration form, Prometheus won't have any issue with that. You'll have one time series that have hundred of thousand of values: one value for each user that registered (how long it took). You'll be able to graph that metric over time, get the p95, etc.

If you want to add *some* cardinality to it, you could add a label like `region`: US-east, Asia-Pacific, etc. You'll be able to graph all regions and compare them or maybe group them all. The number of regions is probably low (<10) and it is certainly bound. If on AWS, there's a fixed number of regions and they don't change much over time. Sure, AWS might add, remove or rename a region, but that's not changing by the minute and there's not thousands of them.

So to come back to what Prometheus advise against: you should not create high-cardinality labels. You should not add a `user_id` label to your `registration_complete` metric. If you do so, you'll have hundred of thousands of *different* time series, one for each user!, and they will all have only one data point. That's really the worst case scenario.

In this case, in order to graph the `registration_complete` metric over all labels, Prometheus will have to query all the individual time series (the X thousands of them) and aggregate all of them.

You said you come from a SQL background, so I'll attempt an analogy. Unique combination of key-value label pairs creating new time series in Prometheus amounts to individual tables in SQL. Having a label `user_id` equals to having one table per user\_id.

Note: not all TSDB work the same way and I can't speak for them all.


---

