<p>Your awk command is inside double quotes (<code>""</code>) which means that all of the awk field variables (e.g. <code>$3</code>) are being interpreted by the shell. Since <code>$3</code> is not set in the shell, your awk is trying to compare an empty string and that's why you get the error:</p>

<pre><code>awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
</code></pre>

<p>Note how the error message shows <code>if( != chrM</code> and not <code>if($3 != chrM</code>. Since you <em>need</em> to use double quotes here because you need both an outer (for <code>parallel</code>) and an inner (for awk) quote, and single quotes can't be nested, you will have to escape the awk variables:</p>

<pre><code>ls *.bam | parallel "samtools view -h {} | 
    awk '{if(\$3 != "chrM" &amp;&amp; \$3 != "chrUn"){print \$0}}' | samtools view -Shb {.}.bam" 
</code></pre>

CC BY-SA 4.0
2019-04-16T11:01:44.597Z
looks like i need to clear my fundamental  awk concepts let me run and will update you. now i ran your code i get output like this "�&Ct�wh���x�Y#��J�f���{/.i��7��%M`�������`C�̛�<kbb�������B��f����<#?���
                                                                                        :P�y���
u��O��r.�_2ؑ�xu�zl�,xϭ"2�*�Uo�ȩ�@ayi���
/0���/���۶�
               �>�Xh� ��Ί�5h�f~���euSa1����;5��{���R�3L����p%��[<�1)M�y!�w��#���_�`�V5rj��v�}�
S-���y)�Nμ4��Jӕ�D���*n�1��ޥ�g�����y���䒋�N��)���<:���mʲ�溎[�ʘ���K��[punit@localhost chrm]$ 62;c62;c62;c62;c^C
"
kcm
CC BY-SA 4.0
2019-04-16T11:12:02.113Z
@krushnachChandra well yes, of course. You are printing a bam file which is compressed binary data. What were you expecting to see? If you want sam output, remove the `b` from the options you give to `samtools view`. The `b` makes it print bam output.
terdon
CC BY-SA 4.0
2019-04-16T11:14:58.687Z
so i idea is after filtering the mitochondrial read and unassembled one i want  bam output written to file with bam format which will be using for downstream analysis.I don't want to print it on the terminal . tried it with -o to write the output seems like i couldn;t make it work..
kcm
CC BY-SA 4.0
2019-04-16T11:19:00.160Z
@krushnachChandra `-o` should work. Also, you could redirect in the parallel command: `ls *.bam | parallel "samtools view -h {} | awk '{if(\$3 != "chrM" && \$3 != "chrUn"){print \$0}}' | samtools view -Shb  > {}.bam"`. But as I said under your question, the new error is completely unrelated. You probably have a malformed bam file. But please ask a new question about that. Even better, run `for bam in *bam; do samtools view -h "$bam" | head -n1 || echo "$bam file is corrupted"; done` That should help you find the bad file.
terdon
CC BY-SA 4.0
2019-04-16T11:22:23.410Z
will run it and see
kcm
CC BY-SA 4.0
2019-04-16T12:31:23.617Z
its working ,the output is written to bam file
kcm
CC BY-SA 4.0
2019-09-16T07:03:38.737Z
i have accepted the answer sorry i missed it to mark since i saw this thread active again .
kcm
CC BY-SA 4.0
2019-04-16T10:54:04.467Z
2019-04-16T10:54:04.467Z
terdon
<p>This is the one i used for a single bam file to filter its mitochondrial as well as unassembled read</p>
<pre><code>samtools view -h SRR2920466.bam | awk '{if(<span class="math-container">$3!= "chrM" &amp;&amp; $</span>3 != &quot;chrUn&quot;){print $0}}' | samtools view -Shb - &gt; this.filter.bam 
</code></pre>
<p>Is there any way i can do it on multiple files with parallel as i do to index and sort bam files using samtools .</p>
<p>So i tried this ,which is of course not working as its not able to pipe properly because it shows me error regarding additional fonts</p>
<pre><code>ls *.bam | parallel &quot;samtools view -h {} | awk '{if(<span class="math-container">$3 != "chrM" &amp;&amp; $</span>3 != &quot;chrUn&quot;){print $0}}'| samtools view -Shb {.}.bam&quot; 
</code></pre>
<p>Any help with my above codes? As i get it there is wrong with my pipe .</p>
<p>After updating the samtools to version 1.9 i ran the small script</p>
<p>i get this error</p>
<pre><code>awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                    ^ unterminated regexp
awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                                   ^ unexpected newline or end of string
awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                    ^ unterminated regexp
awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                                   ^ unexpected newline or end of string
awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                    ^ unterminated regexp
awk: cmd. line:1: {if( != chrM &amp;&amp;  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                                   ^ unexpected newline or end of string
</code></pre>
<p>I have 30 bam files i tried with 2 to see if its working or not , after the suggestion of @WouterDeCoster i tried the above one after updating samtools . the basic samtool with egrep works but for multiple file i couldn;t do it.</p>

<awk><shell>
removing mitochondrial read and unassembled "random" from multiple bam files
# removing mitochondrial read and unassembled "random" from multiple bam files

(Asked by: username_0 on 2019-04-16)

This is the one i used for a single bam file to filter its mitochondrial as well as unassembled read
samtools view -h SRR2920466.bam | awk '{if($3!= "chrM" && $3 != "chrUn"){print $0}}' | samtools view -Shb - > this.filter.bam 

Is there any way i can do it on multiple files with parallel as i do to index and sort bam files using samtools .
So i tried this ,which is of course not working as its not able to pipe properly because it shows me error regarding additional fonts
ls *.bam | parallel "samtools view -h {} | awk '{if($3 != "chrM" && $3 != "chrUn"){print $0}}'| samtools view -Shb {.}.bam" 

Any help with my above codes? As i get it there is wrong with my pipe .
After updating the samtools to version 1.9 i ran the small script
i get this error
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                    ^ unterminated regexp
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                                   ^ unexpected newline or end of string
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                    ^ unterminated regexp
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                                   ^ unexpected newline or end of string
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                    ^ unterminated regexp
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                                   ^ unexpected newline or end of string

I have 30 bam files i tried with 2 to see if its working or not , after the suggestion of @WouterDeCoster i tried the above one after updating samtools . the basic samtool with egrep works but for multiple file i couldn;t do it.

**Tags:** awk, shell

---

## 1 answers

### Answer by username_1 on 2019-04-16. Score: 1 | Accepted answer

Your awk command is inside double quotes ("") which means that all of the awk field variables (e.g. $3) are being interpreted by the shell. Since $3 is not set in the shell, your awk is trying to compare an empty string and that's why you get the error:
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error

Note how the error message shows if( != chrM and not if($3 != chrM. Since you need to use double quotes here because you need both an outer (for parallel) and an inner (for awk) quote, and single quotes can't be nested, you will have to escape the awk variables:
ls *.bam | parallel "samtools view -h {} | 
    awk '{if(\$3 != "chrM" && \$3 != "chrUn"){print \$0}}' | samtools view -Shb {.}.bam"

> **Comments:**
> **username_0**: looks like i need to clear my fundamental  awk concepts let me run and will update you. now i ran your code i get output like this "�&Ct�wh���x�Y#��J�f���{/.i��7��%M`�������`C�̛�<kbb�������B��f����<#?���                                                                                         :P�y��� u��O��r.�_2ؑ�xu�zl�,xϭ"2�*�Uo�ȩ�@ayi��� /0���/���۶�                �>�Xh� ��Ί�5h�f~���euSa1����;5��{���R�3L����p%��[<�1)M�y!�w��#���_�`�V5rj��v�}� S-���y)�Nμ4��Jӕ�D���*n�1��ޥ�g�����y���䒋�N��)���<:���mʲ�溎[�ʘ���K��[punit@localhost chrm]$ 62;c62;c62;c62;c^C "  
> **username_1**: @krushnachChandra well yes, of course. You are printing a bam file which is compressed binary data. What were you expecting to see? If you want sam output, remove the `b` from the options you give to `samtools view`. The `b` makes it print bam output.  
> **username_0**: so i idea is after filtering the mitochondrial read and unassembled one i want  bam output written to file with bam format which will be using for downstream analysis.I don't want to print it on the terminal . tried it with -o to write the output seems like i couldn;t make it work..  
> **username_1**: @krushnachChandra `-o` should work. Also, you could redirect in the parallel command: `ls *.bam | parallel "samtools view -h {} | awk '{if(\$3 != "chrM" && \$3 != "chrUn"){print \$0}}' | samtools view -Shb  > {}.bam"`. But as I said under your question, the new error is completely unrelated. You probably have a malformed bam file. But please ask a new question about that. Even better, run `for bam in *bam; do samtools view -h "$bam" | head -n1 || echo "$bam file is corrupted"; done` That should help you find the bad file.  
> **username_0**: will run it and see  
> **username_0**: its working ,the output is written to bam file  
> **username_0**: i have accepted the answer sorry i missed it to mark since i saw this thread active again .  

---

