<p>I am facing an issue when running snakemake with a container and conda.
Let me explain better.
I am developing a workflow (locally) in snakemake, where some rules use a conda env (defined in /envs/env.yaml) and other rules use wrappers (so they also are based on conda envs).</p>
<p>An example here:</p>
<pre><code>rule clean_spike:
    input:
        sample_ref=&quot;{}results/bam/{{id}}.tmp.bam&quot;.format(outdir),
        sample_spike=&quot;{}results/bam_spike/{{id}}_spike.tmp.bam&quot;.format(outdir),
        sample_ref_index=&quot;{}results/bam/{{id}}.tmp.bam.bai&quot;.format(outdir),
        sample_spike_index=&quot;{}results/bam_spike/{{id}}_spike.tmp.bam.bai&quot;.format(outdir),
    output:
        sample_ref=&quot;{}results/bam/{{id}}.clean.bam&quot;.format(outdir),
        sample_spike=&quot;{}results/bam_spike/{{id}}_spike.clean.bam&quot;.format(outdir),
    conda:
        &quot;../envs/pysam.yaml&quot;
    log:
        &quot;{}results/logs/spike/{{id}}.removeSpikeDups&quot;.format(outdir),
    script:
        &quot;../scripts/remove_spikeDups.py&quot;
</code></pre>
<p>Locally, the pipeline runs smoothly.</p>
<p>Now, I wanted to test the pipeline on another machine (server from the institute), and so I run the command</p>
<p><code>snakemake --containerize &gt; Dockerfile</code></p>
<p>I got a dockerfile and  I generated an image that was uploaded on dockerhub <a href="https://hub.docker.com/repository/docker/davidebrex/chiprxsnakemake/general" rel="nofollow noreferrer">here</a>.</p>
<p>At this point, I copied the workflow on the other machine, I added to the Snakefile the line:</p>
<p><code>containerized: &quot;docker://davidebrex/chiprxsnakemake:latest&quot;</code></p>
<p>and tried to run the pipeline with: <code>snakemake -c 10 --use-conda --use-singularity.</code>
Ideally, snakemake should run the conda envs within the docker container for each rule</p>
<p>The problem is that i get an error related to the fact that conda is missing..
I checked the docker image and I can run conda and see the installed envs.</p>
<p><strong>Any suggestions about what is going on?</strong></p>
<p>Thank you!</p>
<p>This is the error I get:</p>
<pre><code>/usr/bin/bash: conda: command not found
Traceback (most recent call last):
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/__init__.py&quot;, line 792, in snakemake
    success = workflow.execute(
              ^^^^^^^^^^^^^^^^^
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py&quot;, line 1246, in execute
    raise e
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py&quot;, line 1242, in execute
    success = self.scheduler.schedule()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py&quot;, line 636, in schedule
    self.run(runjobs)
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py&quot;, line 685, in run
    executor.run_jobs(
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&quot;, line 158, in run_jobs
    self.run(
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&quot;, line 582, in run
    future = self.run_single_job(job)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&quot;, line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&quot;, line 623, in job_args_and_prepare
    self.workflow.conda_base_path,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py&quot;, line 456, in conda_base_path
    return Conda().prefix_path
           ^^^^^^^
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/deployment/conda.py&quot;, line 653, in __init__
    shell.check_output(self._get_cmd(&quot;conda info --json&quot;), text=True)
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/shell.py&quot;, line 61, in check_output
    return sp.check_output(cmd, shell=True, executable=executable, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py&quot;, line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py&quot;, line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'conda info --json' returned non-zero exit status 127.
</code></pre>
<p>I tried to change the snakemake command and remove --use-conda but then I get an error related to the fact that the tools are missing.
I checked out other questions on stackoverflow but they were not helpful in my case (like this one: <a href="https://stackoverflow.com/questions/64000711/how-to-use-singularity-and-conda-wrappers-in-snakemake">here</a>).</p>

CC BY-SA 4.0
2023-10-26T15:20:44.653Z
I have zero experience with `--containerize`, but "I checked the docker image and I can run conda" versus "conda: command not found" when running the workflow really smells like a conda shell setup issue to to me, like people run into when trying to call conda in a script versus interactively.  (No idea where to point you from there, though!)
Jesse
CC BY-SA 4.0
2023-10-30T10:03:11.097Z
Thank you. At the end I changed snakemake version and it worked..quite weird
DavideBrex
CC BY-SA 4.0
2023-12-07T19:11:29.807Z
Actually now i still get the same error, I also opened an issue on snakemake github, I am clueless
DavideBrex
<python><snakemake><conda><singularity>
Error in snakemake when running conda envs in container
# Error in snakemake when running conda envs in container

(Asked by: username_0 on 2023-10-26)

I am facing an issue when running snakemake with a container and conda.
Let me explain better.
I am developing a workflow (locally) in snakemake, where some rules use a conda env (defined in /envs/env.yaml) and other rules use wrappers (so they also are based on conda envs).
An example here:
rule clean_spike:
    input:
        sample_ref="{}results/bam/{{id}}.tmp.bam".format(outdir),
        sample_spike="{}results/bam_spike/{{id}}_spike.tmp.bam".format(outdir),
        sample_ref_index="{}results/bam/{{id}}.tmp.bam.bai".format(outdir),
        sample_spike_index="{}results/bam_spike/{{id}}_spike.tmp.bam.bai".format(outdir),
    output:
        sample_ref="{}results/bam/{{id}}.clean.bam".format(outdir),
        sample_spike="{}results/bam_spike/{{id}}_spike.clean.bam".format(outdir),
    conda:
        "../envs/pysam.yaml"
    log:
        "{}results/logs/spike/{{id}}.removeSpikeDups".format(outdir),
    script:
        "../scripts/remove_spikeDups.py"

Locally, the pipeline runs smoothly.
Now, I wanted to test the pipeline on another machine (server from the institute), and so I run the command
snakemake --containerize > Dockerfile
I got a dockerfile and  I generated an image that was uploaded on dockerhub here.
At this point, I copied the workflow on the other machine, I added to the Snakefile the line:
containerized: "docker://davidebrex/chiprxsnakemake:latest"
and tried to run the pipeline with: snakemake -c 10 --use-conda --use-singularity.
Ideally, snakemake should run the conda envs within the docker container for each rule
The problem is that i get an error related to the fact that conda is missing..
I checked the docker image and I can run conda and see the installed envs.
Any suggestions about what is going on?
Thank you!
This is the error I get:
/usr/bin/bash: conda: command not found
Traceback (most recent call last):
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/__init__.py", line 792, in snakemake
    success = workflow.execute(
              ^^^^^^^^^^^^^^^^^
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 1246, in execute
    raise e
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py", line 685, in run
    executor.run_jobs(
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
    self.run(
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py", line 582, in run
    future = self.run_single_job(job)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py", line 623, in job_args_and_prepare
    self.workflow.conda_base_path,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 456, in conda_base_path
    return Conda().prefix_path
           ^^^^^^^
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/deployment/conda.py", line 653, in __init__
    shell.check_output(self._get_cmd("conda info --json"), text=True)
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/shell.py", line 61, in check_output
    return sp.check_output(cmd, shell=True, executable=executable, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'conda info --json' returned non-zero exit status 127.

I tried to change the snakemake command and remove --use-conda but then I get an error related to the fact that the tools are missing.
I checked out other questions on stackoverflow but they were not helpful in my case (like this one: here).

**Tags:** python, snakemake, conda, singularity
> **Comments:**
> **username_1**: I have zero experience with `--containerize`, but "I checked the docker image and I can run conda" versus "conda: command not found" when running the workflow really smells like a conda shell setup issue to to me, like people run into when trying to call conda in a script versus interactively.  (No idea where to point you from there, though!)  
> **username_0**: Thank you. At the end I changed snakemake version and it worked..quite weird  
> **username_0**: Actually now i still get the same error, I also opened an issue on snakemake github, I am clueless  

---

## 0 answers

