<p>Never mind, I figured it out. As it turned out I had to write to file like so and extra quotes <code>(&quot;\'&quot;)</code> around samples were not needed. Also shell=True does not work well with lists, only strings:</p>
<pre class="lang-py prettyprint-override"><code>def remove_individuals(chr_samples):
    my_env = os.environ.copy()
    my_env['PATH'] = f'/home/aydar/bin/bcftools_1.11/bin/:{my_env[&quot;PATH&quot;]}'
    def remove_indiv(chr, indiv):
        in_vcf = f'data.filtered-updated-{chr}.vcf.gz'
        out_vcf = f'data.filtered-updated-{chr}-faulty-samp-rem.vcf.gz'
        drop_samples = &quot;^&quot;+&quot;,&quot;.join(indiv)
        out = subprocess.run(
            ['bcftools', 'view', in_vcf, 
             '--samples', drop_samples, '-Oz'],
            env=my_env,
            capture_output=True
        )
        with open(out_vcf, 'wb') as f:
            f.write(out.stdout)
    chr_samples.apply(lambda x: remove_indiv(x.index[0], list(x.values[0])))
</code></pre>
<p>Remember to use <code>'wb'</code> if you like me and use <code>Oz</code> output option, because it returns bgzipped file.</p>

CC BY-SA 4.0
2023-09-01T12:59:45.103Z
thanks for accepting your answer and please feel free to ask a question any time here.
M__
CC BY-SA 4.0
2023-08-27T21:44:24.477Z
2023-08-28T17:08:32.300Z
2023-08-28T17:08:32.300Z
YKY
<p>I am trying to remove samples from a chromosome vcf file. I wrote a function that takes chromosome number and a list of samples to remove. When I try to run bcftools using subprocess module it only runs bcftools, as if I was running <code>subprocess.run([bcftools], shell=True, env=my_env)</code> instead of:</p>
<pre class="lang-py prettyprint-override"><code>subprocess.run(['bcftools', 'view', in_vcf, '--samples', drop_samples, '-Oz', '&gt;', out_vcf], shell=True, env=my_env)
</code></pre>
<p>The function:</p>
<pre class="lang-py prettyprint-override"><code>def remove_individuals(chr_samples):
    my_env = os.environ.copy()
    my_env['PATH'] = f'/home/aydar/bin/bcftools_1.11/bin/:{my_env[&quot;PATH&quot;]}'
    def remove_indiv(chr, indiv):
        in_vcf = f'data.filtered-updated-{chr}.vcf.gz'
        out_vcf = f'data.filtered-updated-{chr}-faulty-samp-rem.vcf.gz'
        drop_samples = &quot;\'^&quot;+&quot;,&quot;.join(indiv)+&quot;\'&quot;
        proc = subprocess.run(
            ['bcftools', 'view', in_vcf, 
             '--samples', drop_samples, '-Oz', '&gt;', out_vcf],
            shell=True,
            env=my_env
        )
        print(proc.args)
    chr_samples.apply(lambda x: remove_indiv(x.index[0], list(x.values[0])))
</code></pre>
<p>function input:</p>
<pre><code>13,[2500000075_2500000075,2000000171_2000000171,1800000179_1800000179,330134048_330134048]
</code></pre>
<p>When I run the same command in the terminal everything works.</p>
<p>The command I test with in terminal:</p>
<pre><code>bcftools view data.filtered-updated-13.vcf.gz --samples '^2500000075_2500000075,2000000171_2000000171,1800000179_1800000179,330134048_330134048' -Oz &gt; out_test_vcf.gz
</code></pre>

CC BY-SA 4.0
2023-08-27T22:22:22.920Z
Thanks for your question and answer. Using `popen` is the standard approach here than using `run`. Anyway could you very kindly mark your answer as "accepted"? This help site stats and prevents the question from being re-introduced as unresolved later.
M__
<python><bcftools>
Can not launch bcftools using python's subprocess module, as it only accepts first command of commands list
# Can not launch bcftools using python's subprocess module, as it only accepts first command of commands list

(Asked by: username_0 on 2023-08-27)

I am trying to remove samples from a chromosome vcf file. I wrote a function that takes chromosome number and a list of samples to remove. When I try to run bcftools using subprocess module it only runs bcftools, as if I was running subprocess.run([bcftools], shell=True, env=my_env) instead of:
subprocess.run(['bcftools', 'view', in_vcf, '--samples', drop_samples, '-Oz', '>', out_vcf], shell=True, env=my_env)

The function:
def remove_individuals(chr_samples):
    my_env = os.environ.copy()
    my_env['PATH'] = f'/home/aydar/bin/bcftools_1.11/bin/:{my_env["PATH"]}'
    def remove_indiv(chr, indiv):
        in_vcf = f'data.filtered-updated-{chr}.vcf.gz'
        out_vcf = f'data.filtered-updated-{chr}-faulty-samp-rem.vcf.gz'
        drop_samples = "\'^"+",".join(indiv)+"\'"
        proc = subprocess.run(
            ['bcftools', 'view', in_vcf, 
             '--samples', drop_samples, '-Oz', '>', out_vcf],
            shell=True,
            env=my_env
        )
        print(proc.args)
    chr_samples.apply(lambda x: remove_indiv(x.index[0], list(x.values[0])))

function input:
13,[2500000075_2500000075,2000000171_2000000171,1800000179_1800000179,330134048_330134048]

When I run the same command in the terminal everything works.
The command I test with in terminal:
bcftools view data.filtered-updated-13.vcf.gz --samples '^2500000075_2500000075,2000000171_2000000171,1800000179_1800000179,330134048_330134048' -Oz > out_test_vcf.gz

**Tags:** python, bcftools
> **Comments:**
> **username_1**: Thanks for your question and answer. Using `popen` is the standard approach here than using `run`. Anyway could you very kindly mark your answer as "accepted"? This help site stats and prevents the question from being re-introduced as unresolved later.  

---

## 1 answers

### Answer by username_0 on 2023-08-27. Score: 3 | Accepted answer

Never mind, I figured it out. As it turned out I had to write to file like so and extra quotes ("\'") around samples were not needed. Also shell=True does not work well with lists, only strings:
def remove_individuals(chr_samples):
    my_env = os.environ.copy()
    my_env['PATH'] = f'/home/aydar/bin/bcftools_1.11/bin/:{my_env["PATH"]}'
    def remove_indiv(chr, indiv):
        in_vcf = f'data.filtered-updated-{chr}.vcf.gz'
        out_vcf = f'data.filtered-updated-{chr}-faulty-samp-rem.vcf.gz'
        drop_samples = "^"+",".join(indiv)
        out = subprocess.run(
            ['bcftools', 'view', in_vcf, 
             '--samples', drop_samples, '-Oz'],
            env=my_env,
            capture_output=True
        )
        with open(out_vcf, 'wb') as f:
            f.write(out.stdout)
    chr_samples.apply(lambda x: remove_indiv(x.index[0], list(x.values[0])))

Remember to use 'wb' if you like me and use Oz output option, because it returns bgzipped file.

> **Comments:**
> **username_1**: thanks for accepting your answer and please feel free to ask a question any time here.  

---

