<p>I see two way to solve this problem:</p>

<ol>
<li><p>Check your logs often and ensure there was an access with code 200 in the last N seconds/minutes and no code 5xx meaning there's a server side error. (the often should be coherent with how long you are ok with no entries)</p></li>
<li><p>Keep an active check but:</p>

<ul>
<li>use a HEAD request (so there's less data to return by the server) </li>
<li>use a custom user agent for the check </li>
<li>configure your log exporter to ignore this user agent.</li>
<li>take a larger check interval, 1 second is probably over obsessive </li>
</ul></li>
</ol>

<p>Depending on how often your sites are called you may wish to mix both methods to have a proper check even if there's no traffic from real clients.</p>

CC BY-SA 4.0
2018-06-07T09:23:43.323Z
2018-06-07T09:23:43.323Z
Tensibai
<p>One possibility would be to monitor the responses to requests from other site users. For example I'd check if there were <code>200 OK</code> responses and skip hitting the server is such responses were sent in the most recent X minutes (up to you to determine which time interval is best fit for your application). </p>

<p>Maybe checking on the (server side) errors as well and hitting the server as well if a sudden change in the error occurrence rate is detected - just to be sure.</p>

CC BY-SA 4.0
2018-06-07T03:32:01.380Z
2018-06-07T03:32:01.380Z
Dan Cornilescu
<p>While I like the other great answers already here, I'd like to add:</p>

<h1>use GET against a different port</h1>

<p>You could have your app listen on a different port which isn't logged.  <code>haproxy</code> and such are happy to check for health on port A for a service that is live on port B.</p>

<h1>don't log canary checks</h1>

<p>You could designate certain data items for monitoring and other tests to retrieve and not log requests against those designated items.</p>

<h1>set a lower TTL for canary entries</h1>

<p>It is a bit scary to cut off logging, so maybe you could delete the entries related to logging more aggressively since it is going into a database?  This avoids checking a code path that turns out to not fail when the user experience is failing.</p>

<h1>end-to-end checks are pretty important</h1>

<p>Having something that is able to</p>

<ul>
<li>see whether it can get to you</li>
<li>and whether your thing is doing its thing</li>
<li>and whether your thing can talk to everything it needs</li>
</ul>

<p>often ends up being pretty vital to detect issues that you wouldn't catch by focusing on any one part of this.  Paying the price for logging is usually negligible compared to the value and peace of mind you get from end-to-end checks.</p>

CC BY-SA 4.0
2018-06-12T18:09:59.823Z
2018-06-12T18:09:59.823Z
chicks
<p>You can use <a href="https://uptimerobot.com/" rel="nofollow noreferrer">Uptime Robot</a>, it offers a very good free layer (up to 50 sites free).</p>

<p>You can configure your sites (they are called monitors) in many ways. A simple <code>GET</code> (your case), <code>ping</code> and more. In case you want to have an idea, the screen to add a monitor look like this:</p>

<p><a href="https://i.sstatic.net/6y06j.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/6y06j.png" alt="enter image description here"></a></p>

<p>You can also configure your email to receive alerts, it has more options but I haven't explored which one of them are on the free tier, but in the other options you can find SMS, slack, hipchat and more.</p>

<p>It has a very good dashboard (I never use it though, I just get the email notifications) where you can see the status of the health checks and the type of error if it's the case.</p>

<p><a href="https://i.sstatic.net/U3GUa.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/U3GUa.png" alt="enter image description here"></a></p>

<p><a href="https://i.sstatic.net/yaVeH.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/yaVeH.png" alt="enter image description here"></a></p>

<p>The only downside on the free tier is the five minutes interval, I would prefer to use a two or three minutes interval but most of the sites don't need one second accuracy on the health check anyway.</p>

CC BY-SA 4.0
2018-06-07T14:12:17.447Z
I fail to understand how this would avoid hitting the service with get request and pilling up 'monitoring' logs...
Tensibai
CC BY-SA 4.0
2018-06-07T15:16:06.837Z
@Tensibai fair enough, my thinking behind this approach is he doesn't want to hit his service every second, generating a lot of noise. With this approach you are still getting some logs but every 5 minutes, that is 300 times less logs. I could be wrong and this solution may not fit his needs, but anyway I just wanted to give him another option so he can choose better.
Gepser Hoil
CC BY-SA 4.0
2018-06-07T04:10:58.040Z
2018-06-07T04:10:58.040Z
Gepser Hoil
<p>One method is that you do TCP-only monitoring. In this scenario, you establish a three way handshake (or use tcp-half-open monitoring in which no connection is established - you merely look for an [syn, ack] to your [syn], and don't [ack] the [syn, ack] back) and then close the connection assuming it is the web server listening on that port.</p>

<p>A better option however, is to figure out how to <a href="https://www.howtoforge.com/setenvif_apache2" rel="nofollow noreferrer">exclude your</a> health <a href="https://bjornjohansen.no/exclude-requests-from-nginx-access-log" rel="nofollow noreferrer">monitor traffic</a> from logging, though I am unable to give specifics on  doing that with rails/mongodb logger.</p>

CC BY-SA 4.0
2018-06-10T15:32:31.717Z
2018-06-10T15:32:31.717Z
James Shewey
<p>I am looking for a solution to monitor the health (<code>DOWN</code> or <code>UP</code>) of my web pages.
Let's say I have three services,</p>

<pre><code>site.com/data
site.com/user
site.com/id
</code></pre>

<p>Now, what I can do is that I can create a <code>GET</code> request which will be served every second and check if <code>response status code</code> is <code>200 OK</code>. If it is anything else than <code>200 OK</code>, then it will trigger an alert. But, I don't want to hit the server continuously, as it will create unnecessary logs. I am using a <code>mongodb logger</code>. What is the recommended way to achieve this?</p>

CC BY-SA 4.0
2018-06-07T05:56:14.777Z
Monitoring also depends on your deployment model as well. Most deployments in cloud supports health checks and liveness checks.
Here_2_learn
<monitoring>
Health monitoring of web pages without using GET check
Dinesh Ahuja
# Health monitoring of web pages without using GET check

(Asked by: username_0 on 2018-06-06)

I am looking for a solution to monitor the health (DOWN or UP) of my web pages.
Let's say I have three services,
site.com/data
site.com/user
site.com/id

Now, what I can do is that I can create a GET request which will be served every second and check if response status code is 200 OK. If it is anything else than 200 OK, then it will trigger an alert. But, I don't want to hit the server continuously, as it will create unnecessary logs. I am using a mongodb logger. What is the recommended way to achieve this?

**Tags:** monitoring

> **Comments:**
> **username_1**: Monitoring also depends on your deployment model as well. Most deployments in cloud supports health checks and liveness checks.  

---

## 5 answers

### Answer by username_2 on 2018-06-07. Score: 5

I see two way to solve this problem:

1. Check your logs often and ensure there was an access with code 200 in the last N seconds/minutes and no code 5xx meaning there's a server side error. (the often should be coherent with how long you are ok with no entries)
2. Keep an active check but:

   * use a HEAD request (so there's less data to return by the server)
   * use a custom user agent for the check
   * configure your log exporter to ignore this user agent.
   * take a larger check interval, 1 second is probably over obsessive

Depending on how often your sites are called you may wish to mix both methods to have a proper check even if there's no traffic from real clients.


---

### Answer by username_3 on 2018-06-07. Score: 1

One possibility would be to monitor the responses to requests from other site users. For example I'd check if there were `200 OK` responses and skip hitting the server is such responses were sent in the most recent X minutes (up to you to determine which time interval is best fit for your application).

Maybe checking on the (server side) errors as well and hitting the server as well if a sudden change in the error occurrence rate is detected - just to be sure.


---

### Answer by username_4 on 2018-06-12. Score: 1

While I like the other great answers already here, I'd like to add:

# use GET against a different port

You could have your app listen on a different port which isn't logged. `haproxy` and such are happy to check for health on port A for a service that is live on port B.

# don't log canary checks

You could designate certain data items for monitoring and other tests to retrieve and not log requests against those designated items.

# set a lower TTL for canary entries

It is a bit scary to cut off logging, so maybe you could delete the entries related to logging more aggressively since it is going into a database? This avoids checking a code path that turns out to not fail when the user experience is failing.

# end-to-end checks are pretty important

Having something that is able to

* see whether it can get to you
* and whether your thing is doing its thing
* and whether your thing can talk to everything it needs

often ends up being pretty vital to detect issues that you wouldn't catch by focusing on any one part of this. Paying the price for logging is usually negligible compared to the value and peace of mind you get from end-to-end checks.


---

### Answer by username_5 on 2018-06-07. Score: 0

You can use [Uptime Robot](https://uptimerobot.com/), it offers a very good free layer (up to 50 sites free).

You can configure your sites (they are called monitors) in many ways. A simple `GET` (your case), `ping` and more. In case you want to have an idea, the screen to add a monitor look like this:

[![enter image description here](https://i.sstatic.net/6y06j.png)](https://i.sstatic.net/6y06j.png)

You can also configure your email to receive alerts, it has more options but I haven't explored which one of them are on the free tier, but in the other options you can find SMS, slack, hipchat and more.

It has a very good dashboard (I never use it though, I just get the email notifications) where you can see the status of the health checks and the type of error if it's the case.

[![enter image description here](https://i.sstatic.net/U3GUa.png)](https://i.sstatic.net/U3GUa.png)

[![enter image description here](https://i.sstatic.net/yaVeH.png)](https://i.sstatic.net/yaVeH.png)

The only downside on the free tier is the five minutes interval, I would prefer to use a two or three minutes interval but most of the sites don't need one second accuracy on the health check anyway.

> **Comments:**
> **username_2**: I fail to understand how this would avoid hitting the service with get request and pilling up 'monitoring' logs...  
> **username_5**: @username_2 fair enough, my thinking behind this approach is he doesn't want to hit his service every second, generating a lot of noise. With this approach you are still getting some logs but every 5 minutes, that is 300 times less logs. I could be wrong and this solution may not fit his needs, but anyway I just wanted to give him another option so he can choose better.  

---

### Answer by username_6 on 2018-06-10. Score: 0

One method is that you do TCP-only monitoring. In this scenario, you establish a three way handshake (or use tcp-half-open monitoring in which no connection is established - you merely look for an [syn, ack] to your [syn], and don't [ack] the [syn, ack] back) and then close the connection assuming it is the web server listening on that port.

A better option however, is to figure out how to [exclude your](https://www.howtoforge.com/setenvif_apache2) health [monitor traffic](https://bjornjohansen.no/exclude-requests-from-nginx-access-log) from logging, though I am unable to give specifics on doing that with rails/mongodb logger.


---

