<p>As far as I am aware, <code>commit</code> and <code>run</code> are the best options here to give you full access to the container as it was when it died.</p>

<p>Ideally, your container would spit out some more useful information when it fails, but that is another topic altogether.</p>

<p>Edit: to expand my answer, if the container is dying right on start, you can also use <code>docker run</code> to specify an alternative <code>--entrypoint</code> and <code>CMD</code>. Generally I'll set this to a loop or something that won't exit on its own. Once you're in the container, you can manually run the steps that are failing, and then inspect the outcome without having to worry about the container exiting.</p>

CC BY-SA 3.0
2017-03-09T02:49:33.433Z
2017-03-11T16:46:04.713Z
2017-03-11T16:46:04.713Z
tayworm
<p>General ways to track why a process in Linux failed are good. One such way is to run a process using <a href="http://man7.org/linux/man-pages/man1/strace.1.html" rel="nofollow noreferrer"><code>strace</code></a> which will tell you the system calls process did and usually point to the reason for a failure.</p>

<p>You can create a <code>Dockerfile</code> that looks something like this:</p>

<pre><code>FROM original_image

RUN apt-get -y update &amp;&amp; apt-get install -y strace

# build with `docker build -t debug_version`
</code></pre>

<p>Then run your new image using <code>docker run debug_version strace original_cmd</code>.</p>

<p>For processes that fork children (and then die) you want to run <code>strace</code> with the <code>-ff</code> option. You can also map some file using <a href="https://docs.docker.com/engine/tutorials/dockervolumes/" rel="nofollow noreferrer">Docker data volumes</a> and use the <code>-o</code> option from <code>strace</code> to write to it. But in general <code>strace</code> will leave output on stdout, which is readable using <code>docker log</code>.</p>

<p>Related <a href="https://serverfault.com/questions/656760/linux-process-terminates-mysteriously">Q: Linux process terminates mysteriously</a></p>

CC BY-SA 3.0
2017-03-07T15:20:09.213Z
This means I still have to `docker commit` my stopped container first to have an image to start from
SztupY
CC BY-SA 3.0
2017-03-07T15:23:26.047Z
You said it dies on start. I assume you have an image then. For those that are stopped, yes a commit is required.
Evgeny Zislis
CC BY-SA 3.0
2017-03-07T15:25:09.273Z
That's only one of the scenarios to get a stopped container
SztupY
CC BY-SA 3.0
2017-03-08T10:43:48.347Z
There is also a package for `strace` in Alpine Linux, https://pkgs.alpinelinux.org/package/edge/main/x86_64/strace. Use the Alpine package manager to install it, `apk install strace`.
Evgeny Zislis
CC BY-SA 3.0
2017-03-07T15:05:58.583Z
2017-03-07T15:12:14.087Z
2017-04-13T12:14:55.217Z
Evgeny Zislis
<p>Sometimes you have to investigate a container, which is stopped, or a container which after starting up dies very quickly and stops.</p>

<p><code>docker exec -ti &lt;id&gt; bash</code> only works on running containers, once it finishes, the bash prompt terminates as well.</p>

<p>With <code>docker start</code> you cannot supply a different command, and if the container dies abruptly again you won't have enough time to get into the container and do your investigations.</p>

<p>We can do <code>docker commit</code>, then <code>docker run</code> on the new image with a different command, but I'm wondering if there are any other alternatives.</p>

<p><strong>Note</strong>: <code>docker logs</code> just returns whatever the apps printed to stdout/stderr. That might not be enough to figure out what the problem was.</p>

CC BY-SA 3.0
2017-03-07T22:15:28.143Z
After a while thinking about it: Docker **main** process ???? As a container aim at running only one process , either  the term 'main' is to be removed, or you're doing something weird (like running an init process) , or you're taking threads as processes... I assume it's option one, but I had to say it because it bothers me
Tensibai
CC BY-SA 3.0
2017-03-08T09:04:44.713Z
@Tensibai sometimes you have to run something like dumb-init, to handle the pid 1 / signalling issue in containers, if your main command cannot handle it itself. There might be also other cases where a docker container runs more than one processes
SztupY
CC BY-SA 3.0
2017-03-08T10:23:01.977Z
Yes, that's what I call weird, mainly because containers have been made to isolate a process. Sometimes containers are not the solution for an app an willing to put everything inside a container is more a path to headaches than anything else.
Tensibai
<docker><containers><post-mortem>
How to investigate a main process that has died in a docker container?
SztupY
# How to investigate a main process that has died in a docker container?

(Asked by: username_0 on 2017-03-07)

Sometimes you have to investigate a container, which is stopped, or a container which after starting up dies very quickly and stops.

`docker exec -ti  bash` only works on running containers, once it finishes, the bash prompt terminates as well.

With `docker start` you cannot supply a different command, and if the container dies abruptly again you won't have enough time to get into the container and do your investigations.

We can do `docker commit`, then `docker run` on the new image with a different command, but I'm wondering if there are any other alternatives.

**Note**: `docker logs` just returns whatever the apps printed to stdout/stderr. That might not be enough to figure out what the problem was.

**Tags:** docker, containers, post-mortem

> **Comments:**
> **username_1**: After a while thinking about it: Docker **main** process ???? As a container aim at running only one process , either  the term 'main' is to be removed, or you're doing something weird (like running an init process) , or you're taking threads as processes... I assume it's option one, but I had to say it because it bothers me  
> **username_0**: @username_1 sometimes you have to run something like dumb-init, to handle the pid 1 / signalling issue in containers, if your main command cannot handle it itself. There might be also other cases where a docker container runs more than one processes  
> **username_1**: Yes, that's what I call weird, mainly because containers have been made to isolate a process. Sometimes containers are not the solution for an app an willing to put everything inside a container is more a path to headaches than anything else.  

---

## 2 answers

### Answer by username_3 on 2017-03-07. Score: 9

General ways to track why a process in Linux failed are good. One such way is to run a process using strace which will tell you the system calls process did and usually point to the reason for a failure.
You can create a Dockerfile that looks something like this:
FROM original_image

RUN apt-get -y update && apt-get install -y strace

# build with `docker build -t debug_version`

Then run your new image using docker run debug_version strace original_cmd.
For processes that fork children (and then die) you want to run strace with the -ff option. You can also map some file using Docker data volumes and use the -o option from strace to write to it. But in general strace will leave output on stdout, which is readable using docker log.
Related Q: Linux process terminates mysteriously

> **Comments:**
> **username_0**: This means I still have to `docker commit` my stopped container first to have an image to start from  
> **username_3**: You said it dies on start. I assume you have an image then. For those that are stopped, yes a commit is required.  
> **username_0**: That's only one of the scenarios to get a stopped container  
> **username_3**: There is also a package for `strace` in Alpine Linux, https://pkgs.alpinelinux.org/package/edge/main/x86_64/strace. Use the Alpine package manager to install it, `apk install strace`.  

---

### Answer by username_2 on 2017-03-09. Score: 3

As far as I am aware, `commit` and `run` are the best options here to give you full access to the container as it was when it died.

Ideally, your container would spit out some more useful information when it fails, but that is another topic altogether.

Edit: to expand my answer, if the container is dying right on start, you can also use `docker run` to specify an alternative `--entrypoint` and `CMD`. Generally I'll set this to a loop or something that won't exit on its own. Once you're in the container, you can manually run the steps that are failing, and then inspect the outcome without having to worry about the container exiting.


---

