<p>Changing logging driver form journald to json has fixed it. 
I edited <code>/etc/sysconfig/docker</code>, and set </p>

<pre><code>OPTIONS='--selinux-enabled --log-driver=json-file --signature-verification=false'
</code></pre>

<p>Then restarted docker on the nodes I needed:</p>

<pre><code>systemctl daemon-reload &amp;&amp; systemctl restart docker
</code></pre>

CC BY-SA 4.0
2019-12-03T12:15:54.753Z
Could you please explain _how_ you changed the logging driver?
Bruce Becker
CC BY-SA 4.0
2019-12-03T12:21:00.847Z
vi /etc/sysconfig/docker: set OPTIONS='--selinux-enabled --log-driver=json-file --signature-verification=false' and then systemctl daemon-reload && systemctl restart docker on the nodes I needed.
anVzdGFub3RoZXJodW1hbg
CC BY-SA 4.0
2019-12-03T12:22:33.893Z
Great! but could you update the answer with this information? Thanks
Bruce Becker
CC BY-SA 4.0
2019-12-03T11:47:00.933Z
2019-12-03T12:53:52.610Z
2019-12-03T12:53:52.610Z
anVzdGFub3RoZXJodW1hbg
<p>I install a fluentd chart on OKD 3.11 with:</p>

<pre><code>helm install kiwigrid/fluentd-elasticsearch \
--name fluentd \
-f ../../tools/fluentd/values.yaml \
--namespace logging
</code></pre>

<p>The pod is created, I can enter in it and I can curl to external elasticsearch successfully.
However the liveness probe keeps failing and the pods on timeout get recreated:</p>

<pre><code>Normal   Killing    10m (x3 over 36m) kubelet node1.qa1.isp.ezbob.com 
Killing container with id docker://fluentd-fluentd-elasticsearch:Container failed liveness probe.. Container will be killed and recreated.
</code></pre>

<p>And I can't find any indexes being created in ES.</p>

<p>The liveness config is the default for the chart:</p>

<pre><code> Liveness:       exec [/bin/sh -c LIVENESS_THRESHOLD_SECONDS=${LIVENESS_THRESHOLD_SECONDS:-300}; STUCK_THRESHOLD_SECONDS=${STUCK_THRESHOLD_SECONDS:-900}; if [ ! -e /var/log/fluentd-buffers ]; then
  exit 1;
fi; touch -d "${STUCK_THRESHOLD_SECONDS} seconds ago" /tmp/marker-stuck; if [ -z "$(find /var/log/fluentd-buffers -type d -newer /tmp/marker-stuck -print -quit)" ]; then
  rm -rf /var/log/fluentd-buffers;
  exit 1;
fi; touch -d "${LIVENESS_THRESHOLD_SECONDS} seconds ago" /tmp/marker-liveness; if [ -z "$(find /var/log/fluentd-buffers -type d -newer /tmp/marker-liveness -print -quit)" ]; then
  exit 1;
fi;
] delay=600s timeout=1s period=60s #success=1 #failure=3
</code></pre>

<p>The OS is CentOS 7.6. I can't get any info about this issue and no trace to catch, it looks like some permission issues on mounting points.</p>

<elastic><openshift><fluentd>
elasticsearch-fluentd helm chart on Openshift 3.11 pod failing
anVzdGFub3RoZXJodW1hbg
# elasticsearch-fluentd helm chart on Openshift 3.11 pod failing

(Asked by: username_0 on 2019-12-02)

I install a fluentd chart on OKD 3.11 with:
helm install kiwigrid/fluentd-elasticsearch \
--name fluentd \
-f ../../tools/fluentd/values.yaml \
--namespace logging

The pod is created, I can enter in it and I can curl to external elasticsearch successfully.
However the liveness probe keeps failing and the pods on timeout get recreated:
Normal   Killing    10m (x3 over 36m) kubelet node1.qa1.isp.ezbob.com 
Killing container with id docker://fluentd-fluentd-elasticsearch:Container failed liveness probe.. Container will be killed and recreated.

And I can't find any indexes being created in ES.
The liveness config is the default for the chart:
 Liveness:       exec [/bin/sh -c LIVENESS_THRESHOLD_SECONDS=${LIVENESS_THRESHOLD_SECONDS:-300}; STUCK_THRESHOLD_SECONDS=${STUCK_THRESHOLD_SECONDS:-900}; if [ ! -e /var/log/fluentd-buffers ]; then
  exit 1;
fi; touch -d "${STUCK_THRESHOLD_SECONDS} seconds ago" /tmp/marker-stuck; if [ -z "$(find /var/log/fluentd-buffers -type d -newer /tmp/marker-stuck -print -quit)" ]; then
  rm -rf /var/log/fluentd-buffers;
  exit 1;
fi; touch -d "${LIVENESS_THRESHOLD_SECONDS} seconds ago" /tmp/marker-liveness; if [ -z "$(find /var/log/fluentd-buffers -type d -newer /tmp/marker-liveness -print -quit)" ]; then
  exit 1;
fi;
] delay=600s timeout=1s period=60s #success=1 #failure=3

The OS is CentOS 7.6. I can't get any info about this issue and no trace to catch, it looks like some permission issues on mounting points.

**Tags:** elastic, openshift, fluentd


---

## 1 answers

### Answer by username_0 on 2019-12-03. Score: 2 | Accepted answer

Changing logging driver form journald to json has fixed it. 
I edited /etc/sysconfig/docker, and set 
OPTIONS='--selinux-enabled --log-driver=json-file --signature-verification=false'

Then restarted docker on the nodes I needed:
systemctl daemon-reload && systemctl restart docker

> **Comments:**
> **username_1**: Could you please explain _how_ you changed the logging driver?  
> **username_0**: vi /etc/sysconfig/docker: set OPTIONS='--selinux-enabled --log-driver=json-file --signature-verification=false' and then systemctl daemon-reload && systemctl restart docker on the nodes I needed.  
> **username_1**: Great! but could you update the answer with this information? Thanks  

---

