<p>Eventually, I chose to use docker daemon in docker as a long running pod and Jenkins agents that have docker client inside triggered on demand. They interact with containerized Docker daemon.</p>
<p>I summarized my choice of <a href="https://www.rokpoto.com/jenkins-docker-in-docker-agent/" rel="nofollow noreferrer">Jenkins Docker in Docker Agent</a>.</p>

CC BY-SA 4.0
2022-12-10T20:39:00.223Z
2022-12-10T20:39:00.223Z
rokpoto.com
<p>There are ways to do it but I've not been able to manage it. I would recommend sidestepping the problem by using <a href="https://github.com/GoogleContainerTools/kaniko" rel="nofollow noreferrer">kaniko</a>. You can add a purpose built kaniko container into your 'build' agent k8s podTemplate and let it handle the container build (and push) part of the workload with no docker daemons or sockets necessary.</p>
<p>It's working well for me and I am also using jenkins via helm on k8s.
Here is a snippet of my helm chart <code>values.yaml</code> agent.podTemplates section.</p>
<p>Note that the 'builder' agent contains 2 containers (actually 3 including the default 'jnlp' container which handles communicating with the jenkins controller) and they share a common workspace:</p>
<pre><code>agent:
  namespace: jenkins
  podTemplates:
    builder: |
    - name: 'builder'
      label: 'builder'
      serviceAccount: jenkins
      nodeUsageMode: EXCLUSIVE
      yamlMergeStrategy: &quot;merge&quot;
      yaml: |-
        apiVersion: v1
        kind: Pod
        metadata:
          name: builder
          namespace: jenkins
        spec:
          serviceAccountName: jenkins
          containers:
          - name: builder
            image: builder:latest
            imagePullPolicy: IfNotPresent
            command: [&quot;sleep&quot;]
            args: [&quot;99d&quot;]
            tty: true
            resources:
              requests:
                cpu: 1
                memory: 1Gi
              limits:
                cpu: 2
                memory: 2Gi
            workingDir: /home/jenkins/agent
          - name: kaniko
            image: gcr.io/kaniko-project/executor:debug
            imagePullPolicy: IfNotPresent
            command: [&quot;sleep&quot;]
            args: [&quot;99d&quot;]
            tty: true
            volumeMounts:
            - name: docker-config
              mountPath: /kaniko/.docker
            workingDir: /home/jenkins/agent
            env:
            - name: DOCKER_REPOSITORY
              value: &quot;private_docker_repository&quot;
          volumes:
          - name: docker-config
            configMap:
              name: docker-config
</code></pre>
<p>Note that I used the <code>yaml: |-</code> field  because it was the only way I could find to mount a volume containing the configMap needed by kaniko for docker authorization, <a href="https://github.com/jenkinsci/kubernetes-plugin#configuration-reference" rel="nofollow noreferrer">configuration reference</a>.
So I also kubectl applied a simple docker-config configMap in the jenkins namespace first before applying this helm chart values.yaml.</p>
<pre><code>apiVersion: v1
kind: ConfigMap
metadata:
  name: docker-config
  namespace: jenkins
data:
  config.json: |-
    {
       &quot;credsStore&quot;: &quot;ecr-login&quot;
    }
</code></pre>
<p>Find more details on what that docker-config needs to contain for you <a href="https://github.com/GoogleContainerTools/kaniko#pushing-to-different-registries" rel="nofollow noreferrer">here</a>.</p>
<p>And then here is an example of using the different containers in a declarative Jenkinsfile script:</p>
<pre><code>pipeline {
  
  agent {
    label 'builder' // Refers to the custom k8s pod template defined in the jenkins helm chart values.yaml
  }

  environment {
    RELEASE_TAG = &quot;1.0.0&quot;
  }
  
  stages {
    stage('Build') {
      steps {
        container('builder') { // refers to the 'builder' container within the 'builder' agent pod
          echo &quot;do a maven build or whatever other steps you need to take here&quot;
        }
        container('kaniko') { // refers to the 'kaniko' container within the 'builder' agent pod sharing the same workspace
          /kaniko/executor --dockerfile `pwd`/dockerfile --context `pwd` --destination=${DOCKER_REPOSITORY}:${RELEASE_TAG}
        }
      }
    }
  }
}
</code></pre>

CC BY-SA 4.0
2022-03-17T12:51:29.493Z
2022-03-17T12:57:17.263Z
2022-03-17T12:57:17.263Z
david_beauchamp
<p>Jenkins is installed using official helm chart on Kubernetes cluster that uses <code>containerd</code> as container runtime.</p>
<p>How to configure containerized agents which will run docker client commands e.g. for building docker images.</p>
<p>There's no docker daemon on Kubernetes nodes. Is there an option to use containerd client which will interact with containerd server on the nodes? The idea is similar to case where docker client inside the container interacts with docker daemon running on the node.</p>
<p>I'm also thinking about below options for Jenkins agents:</p>
<ul>
<li><a href="https://hub.docker.com/_/docker/" rel="nofollow noreferrer">docker daemon in docker</a></li>
<li><a href="https://github.com/nestybox/sysbox" rel="nofollow noreferrer">sysbox containers</a> having docker daemon</li>
</ul>
<p>Is there a best practice?</p>

CC BY-SA 4.0
2022-04-13T14:11:18.780Z
Very similar to this existing question https://devops.stackexchange.com/questions/13179/how-to-run-a-container-from-within-another-container-while-also-mounting-a-volum
Levi
<docker><jenkins><kubernetes><docker-in-docker><containerd>
Running builds requiring docker daemon in Jenkins installed using helm and running as containerd container
rokpoto.com
# Running builds requiring docker daemon in Jenkins installed using helm and running as containerd container

(Asked by: username_0 on 2022-02-08)

Jenkins is installed using official helm chart on Kubernetes cluster that uses `containerd` as container runtime.

How to configure containerized agents which will run docker client commands e.g. for building docker images.

There's no docker daemon on Kubernetes nodes. Is there an option to use containerd client which will interact with containerd server on the nodes? The idea is similar to case where docker client inside the container interacts with docker daemon running on the node.

I'm also thinking about below options for Jenkins agents:

* [docker daemon in docker](https://hub.docker.com/_/docker/)
* [sysbox containers](https://github.com/nestybox/sysbox) having docker daemon

Is there a best practice?

**Tags:** docker, jenkins, kubernetes, docker-in-docker, containerd

> **Comments:**
> **username_1**: Very similar to this existing question https://devops.stackexchange.com/questions/13179/how-to-run-a-container-from-within-another-container-while-also-mounting-a-volum  

---

## 2 answers

### Answer by username_0 on 2022-12-10. Score: 0 | Accepted answer

Eventually, I chose to use docker daemon in docker as a long running pod and Jenkins agents that have docker client inside triggered on demand. They interact with containerized Docker daemon.

I summarized my choice of [Jenkins Docker in Docker Agent](https://www.username_0/jenkins-docker-in-docker-agent/).


---

### Answer by username_2 on 2022-03-17. Score: 1

There are ways to do it but I've not been able to manage it. I would recommend sidestepping the problem by using kaniko. You can add a purpose built kaniko container into your 'build' agent k8s podTemplate and let it handle the container build (and push) part of the workload with no docker daemons or sockets necessary.
It's working well for me and I am also using jenkins via helm on k8s.
Here is a snippet of my helm chart values.yaml agent.podTemplates section.
Note that the 'builder' agent contains 2 containers (actually 3 including the default 'jnlp' container which handles communicating with the jenkins controller) and they share a common workspace:
agent:
  namespace: jenkins
  podTemplates:
    builder: |
    - name: 'builder'
      label: 'builder'
      serviceAccount: jenkins
      nodeUsageMode: EXCLUSIVE
      yamlMergeStrategy: "merge"
      yaml: |-
        apiVersion: v1
        kind: Pod
        metadata:
          name: builder
          namespace: jenkins
        spec:
          serviceAccountName: jenkins
          containers:
          - name: builder
            image: builder:latest
            imagePullPolicy: IfNotPresent
            command: ["sleep"]
            args: ["99d"]
            tty: true
            resources:
              requests:
                cpu: 1
                memory: 1Gi
              limits:
                cpu: 2
                memory: 2Gi
            workingDir: /home/jenkins/agent
          - name: kaniko
            image: gcr.io/kaniko-project/executor:debug
            imagePullPolicy: IfNotPresent
            command: ["sleep"]
            args: ["99d"]
            tty: true
            volumeMounts:
            - name: docker-config
              mountPath: /kaniko/.docker
            workingDir: /home/jenkins/agent
            env:
            - name: DOCKER_REPOSITORY
              value: "private_docker_repository"
          volumes:
          - name: docker-config
            configMap:
              name: docker-config

Note that I used the yaml: |- field  because it was the only way I could find to mount a volume containing the configMap needed by kaniko for docker authorization, configuration reference.
So I also kubectl applied a simple docker-config configMap in the jenkins namespace first before applying this helm chart values.yaml.
apiVersion: v1
kind: ConfigMap
metadata:
  name: docker-config
  namespace: jenkins
data:
  config.json: |-
    {
       "credsStore": "ecr-login"
    }

Find more details on what that docker-config needs to contain for you here.
And then here is an example of using the different containers in a declarative Jenkinsfile script:
pipeline {
  
  agent {
    label 'builder' // Refers to the custom k8s pod template defined in the jenkins helm chart values.yaml
  }

  environment {
    RELEASE_TAG = "1.0.0"
  }
  
  stages {
    stage('Build') {
      steps {
        container('builder') { // refers to the 'builder' container within the 'builder' agent pod
          echo "do a maven build or whatever other steps you need to take here"
        }
        container('kaniko') { // refers to the 'kaniko' container within the 'builder' agent pod sharing the same workspace
          /kaniko/executor --dockerfile `pwd`/dockerfile --context `pwd` --destination=${DOCKER_REPOSITORY}:${RELEASE_TAG}
        }
      }
    }
  }
}


---

