<p>It's possible but after doing many replatforming projects for clients I can't say I would advise it. Have you seen <a href="https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus" rel="nofollow noreferrer">kube-prometheus</a>? It's a one-shot install that will get you 90% of where you need to be for your cluster.</p>

<p>The only way I can advise it would be if you're going to run 5 or more clusters and want central monitoring, but I'm a bigger fan of having a Grafana per cluster.</p>

<p>Edit: Getting started with kube-prometheus is a one command install. <code>kubectl apply -f manifests/</code></p>

CC BY-SA 3.0
2018-04-04T12:24:45.693Z
I did a 15 minute Prometheus install from a Medium article. It was amazing. Still we have an ask to have the monitoring outside the cluster so that when the cluster goes down the monitoring is there. Is there a way to do that? Is the ROI high enough? Or do you still recommend in cluster?
David West
CC BY-SA 3.0
2018-04-04T14:41:36.263Z
@DavidWest What will you monitor? Are you running ETCD on servers off the cluster as well? See when you lose the API servers, you are going to lose monitoring of your services eventually. This could be immediately or over time. This is because Prometheus relies on the API servers for service discovery and internally some meta_tags provided by the API servers for configuration. 

You can use certain POD labels to ensure your Prometheus pod is the "last man standing". I believe the recommended approach is to run Prometheus on top of Kubernetes.
Levi
CC BY-SA 3.0
2018-04-04T15:58:13.360Z
It will be there for capacity planning, but, more importantly, to monitor the cluster health.
David West
CC BY-SA 3.0
2018-04-10T17:41:04.713Z
It seems to be working well in cluster. Plus, it was so easy to do it that way.
David West
CC BY-SA 4.0
2019-02-24T13:30:37.860Z
How do you get the application metrics to Prometheus?
avi
CC BY-SA 4.0
2019-02-24T21:56:06.763Z
@avi You should probably start a new question ;). Are you talking about custom metrics the application may provide? Like how long a certain function is taking or size of a queue? In that case you can add a JSON endpoint in the format Prometheus accepts, add a tag to your pod and then configure Prometheus to scrape that tag. Now you will see the metrics in Prometheus and be able to graph them in Grafana. If you mean container metrics like CPU and Ram, most Prometheus installations on Kubernetes will handle that for you.
Levi
CC BY-SA 4.0
2019-02-25T07:25:53.507Z
@Levi Thanks. I asked a related question here:  https://devops.stackexchange.com/questions/6450/which-libraries-generate-observability-metrics-for-applications-in-the-cluster-t
avi
CC BY-SA 4.0
2018-04-04T12:21:19.590Z
2019-02-24T21:58:59.773Z
2019-02-24T21:58:59.773Z
Levi
<p>We want to start our monitoring off the cluster on a local server.  Is this practical and will it be feasible and reasonable to hook local Prometheus and Grafana installations into a Kubernetes cluster? </p>

<kubernetes><prometheus><grafana>
Local Prometheus and Grafana Installations to Monitor a Kubernetes Cluster
David West
# Local Prometheus and Grafana Installations to Monitor a Kubernetes Cluster

(Asked by: username_0 on 2018-04-03)

We want to start our monitoring off the cluster on a local server. Is this practical and will it be feasible and reasonable to hook local Prometheus and Grafana installations into a Kubernetes cluster?

**Tags:** kubernetes, prometheus, grafana


---

## 1 answers

### Answer by username_1 on 2018-04-04. Score: 3 | Accepted answer

It's possible but after doing many replatforming projects for clients I can't say I would advise it. Have you seen [kube-prometheus](https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus)? It's a one-shot install that will get you 90% of where you need to be for your cluster.

The only way I can advise it would be if you're going to run 5 or more clusters and want central monitoring, but I'm a bigger fan of husername_2ng a Grafana per cluster.

Edit: Getting started with kube-prometheus is a one command install. `kubectl apply -f manifests/`

> **Comments:**
> **username_0**: I did a 15 minute Prometheus install from a Medium article. It was amazing. Still we have an ask to have the monitoring outside the cluster so that when the cluster goes down the monitoring is there. Is there a way to do that? Is the ROI high enough? Or do you still recommend in cluster?  
> **username_1**: @Dusername_2dWest What will you monitor? Are you running ETCD on servers off the cluster as well? See when you lose the API servers, you are going to lose monitoring of your services eventually. This could be immediately or over time. This is because Prometheus relies on the API servers for service discovery and internally some meta_tags provided by the API servers for configuration.   You can use certain POD labels to ensure your Prometheus pod is the "last man standing". I believe the recommended approach is to run Prometheus on top of Kubernetes.  
> **username_0**: It will be there for capacity planning, but, more importantly, to monitor the cluster health.  
> **username_0**: It seems to be working well in cluster. Plus, it was so easy to do it that way.  
> **username_2**: How do you get the application metrics to Prometheus?  
> **username_1**: @username_2 You should probably start a new question ;). Are you talking about custom metrics the application may provide? Like how long a certain function is taking or size of a queue? In that case you can add a JSON endpoint in the format Prometheus accepts, add a tag to your pod and then configure Prometheus to scrape that tag. Now you will see the metrics in Prometheus and be able to graph them in Grafana. If you mean container metrics like CPU and Ram, most Prometheus installations on Kubernetes will handle that for you.  
> **username_2**: @username_1 Thanks. I asked a related question here:  https://devops.stackexchange.com/questions/6450/which-libraries-generate-observability-metrics-for-applications-in-the-cluster-t  

---

