<p>I finally found out the problem: the fluent container needs some time to start up; because it's started with <code>--detach</code> the subsequent containers try to connect before fluentd is actually running, only the 3rd instance of the flask application connects when fluentd is already up. So, delaying the startup of the remaining containers solves my problem.</p>

CC BY-SA 4.0
2019-08-20T19:01:01.967Z
2019-08-20T19:01:01.967Z
user2683038
<p>I am facing a strange problem with logging when I set up a network of three docker containers each running a small flask app.</p>

<p>My python application looks as follows:</p>

<pre><code>from flask import Flask, request
import sys
import logging
import logging.handlers
import os

app = Flask(__name__)

logger = logging.getLogger('Logger')
logger.setLevel(logging.DEBUG)
log_handler = logging.StreamHandler(sys.stdout)
logger.addHandler(log_handler)

service_name=""
if 'SERVICE_NAME' in os.environ:
    service_name = os.environ['SERVICE_NAME']

logger = logging.LoggerAdapter(logger, {"serviceName" : service_name})

@app.route("/ping")
def ping():
    logger.info(f'answering ping request')

    return f'{service_name} OK'

if __name__ == "__main__":
    logger.info("Started")

    app.run(debug=True, host='0.0.0.0')
</code></pre>

<p>I start three instances of this in a docker network behind nginx:</p>

<pre><code>docker network create tNet

docker run -it -d -p 24224:24224 -v `pwd`:/fluentd/etc -e FLUENTD_CONF=fluentd.conf --name=fluentd_logging fluent/fluentd

docker run -d --log-driver=fluentd --name ms1 -e SERVICE_NAME="ms1" --network tNet testflask
docker run -d --log-driver=fluentd --name ms2 -e SERVICE_NAME="ms2" --network tNet testflask
docker run -d --log-driver=fluentd --name ms3 -e SERVICE_NAME="ms3" --network tNet testflask

docker run -d --name=nginx --network tNet --volume `pwd`/nginx.conf:/etc/nginx/conf.d/default.conf -p 2000:2000 -p 3000:3000 nginx
</code></pre>

<p>The nginx configuration is as follows:</p>

<pre><code>upstream app {
    server ms1:5000;
    server ms2:5000;
    server ms3:5000;
}

server {
    listen 2000;

    location / {
        proxy_pass http://app;
    }
}
</code></pre>

<p>My fluent configuration looks as follows:</p>

<pre><code>&lt;source&gt;
    @type forward
&lt;/source&gt;

&lt;match&gt;
    @type stdout
&lt;/match&gt;
</code></pre>

<p>When I use curl I can see that each instance of the service is responding:</p>

<pre><code>curl localhost:2000/ping
ms1 OK
curl localhost:2000/ping
ms2 OK
curl localhost:2000/ping
ms3 OK
</code></pre>

<p>When I however inspect the logs I can only see logs from the 3rd instance of the service:</p>

<pre><code>docker logs &lt;fluent container ID&gt; | grep -Eis "answering ping"

19-08-06 15:34:46.000000000 +0000 be338a3d5aa0: {"container_id":"be338a3d5aa0b8920366fdf1ba2a267082cee47e681adcea16b6c51c9c82cf0d","container_name":"/ms3","source":"stdout","log":"answering ping request"}
</code></pre>

<p>Has anyone an idea? Thank you in advance.</p>

<microservices><logging><flask>
Problem with logging in docker when using fluentd for logging of three flask apps running in a docker network behind an nginx
user2683038
# Problem with logging in docker when using fluentd for logging of three flask apps running in a docker network behind an nginx

(Asked by: username_0 on 2019-08-06)

I am facing a strange problem with logging when I set up a network of three docker containers each running a small flask app.
My python application looks as follows:
from flask import Flask, request
import sys
import logging
import logging.handlers
import os

app = Flask(__name__)

logger = logging.getLogger('Logger')
logger.setLevel(logging.DEBUG)
log_handler = logging.StreamHandler(sys.stdout)
logger.addHandler(log_handler)

service_name=""
if 'SERVICE_NAME' in os.environ:
    service_name = os.environ['SERVICE_NAME']

logger = logging.LoggerAdapter(logger, {"serviceName" : service_name})

@app.route("/ping")
def ping():
    logger.info(f'answering ping request')

    return f'{service_name} OK'

if __name__ == "__main__":
    logger.info("Started")

    app.run(debug=True, host='0.0.0.0')

I start three instances of this in a docker network behind nginx:
docker network create tNet

docker run -it -d -p 24224:24224 -v `pwd`:/fluentd/etc -e FLUENTD_CONF=fluentd.conf --name=fluentd_logging fluent/fluentd

docker run -d --log-driver=fluentd --name ms1 -e SERVICE_NAME="ms1" --network tNet testflask
docker run -d --log-driver=fluentd --name ms2 -e SERVICE_NAME="ms2" --network tNet testflask
docker run -d --log-driver=fluentd --name ms3 -e SERVICE_NAME="ms3" --network tNet testflask

docker run -d --name=nginx --network tNet --volume `pwd`/nginx.conf:/etc/nginx/conf.d/default.conf -p 2000:2000 -p 3000:3000 nginx

The nginx configuration is as follows:
upstream app {
    server ms1:5000;
    server ms2:5000;
    server ms3:5000;
}

server {
    listen 2000;

    location / {
        proxy_pass http://app;
    }
}

My fluent configuration looks as follows:

    @type forward



    @type stdout


When I use curl I can see that each instance of the service is responding:
curl localhost:2000/ping
ms1 OK
curl localhost:2000/ping
ms2 OK
curl localhost:2000/ping
ms3 OK

When I however inspect the logs I can only see logs from the 3rd instance of the service:
docker logs  | grep -Eis "answering ping"

19-08-06 15:34:46.000000000 +0000 be338a3d5aa0: {"container_id":"be338a3d5aa0b8920366fdf1ba2a267082cee47e681adcea16b6c51c9c82cf0d","container_name":"/ms3","source":"stdout","log":"answering ping request"}

Has anyone an idea? Thank you in advance.

**Tags:** microservices, logging, flask


---

## 1 answers

### Answer by username_0 on 2019-08-20. Score: 0 | Accepted answer

I finally found out the problem: the fluent container needs some time to start up; because it's started with `--detach` the subsequent containers try to connect before fluentd is actually running, only the 3rd instance of the flask application connects when fluentd is already up. So, delaying the startup of the remaining containers solves my problem.


---

