<p>You are getting a syntax error because your command is in double quotes. This means that the awk variables you are using, <code>$4</code> and <code>$5</code>, are being expanded by the shell before launching <code>awk</code>, and since they are not defined, they are passed to <code>awk</code> as empty, making your command:</p>
<pre><code>awk -F'\t' -v OFS='\t' '/^[^#]/{-=5}{+=5}1' &quot;
</code></pre>
<p>And that is what gives you the syntax error. If you are 100% sure your file names contain no whitespace and no globbing characters, so you can use <code>ls</code> as you have done (<a href="https://mywiki.wooledge.org/ParsingLs" rel="nofollow noreferrer">which is generally a very bad idea</a>), then this should work:</p>
<pre><code> ls ./*ratio.gff.gz | 
   parallel &quot;zcat {} | head | awk -F'\t' -v OFS='\t' '/^[^#]/{\\\$5-=5}{\\\$4+=5}1' &quot;
</code></pre>
<p>However, that won't change the original files and it isn't saving the output anywhere, so I doubt it's what you really want to do. This command will edit each file, and save the result into a new file with the same name plus <code>edited.gz</code>. So if you start with <code>foo.gff.gz</code>, you will get a new file called <code>foo.gff.gz.edited.gz</code>:</p>
<pre><code>parallel &quot;awk -F'\t' -v OFS='\t' '/^[^#]/{\\\$5-=5}{\\\$4+=5}1' &lt;(zcat {}) | gzip &gt; {}.edited.gz&quot; ::: *gff.gz
</code></pre>
<p>Once you are sure this works as you want, you can rename all the <code>edited.gz</code> files back to the original name, overwriting the originals:</p>
<pre><code>for f in *edited.gz; do mv -- &quot;$f&quot; &quot;${f//.edited.gz}&quot;; done
</code></pre>
<hr />
<p><strong>IMPORTANT</strong>: The solution above is affected by a markdown rendering bug. There should only ever be <em>one</em> <code>\</code>, as can be seen in the source of this answer, and not three. Make sure to replace all occurrences of <code>\\\</code> in this answer with <code>\</code>.</p>

CC BY-SA 4.0
2024-08-07T21:00:26.090Z
Thank you so much, this is exactly what I want to do. Somehow on the server I use (Debian) "\\\" also returns an error ```awk: 1: unexpected character '\' ```, if changed to "\", it worked.   ```parallel "awk -F'\t' -v OFS='\t' '/^[^#]/{\$5-=5}{\$4+=5}1' <(zcat {}) | gzip > {}.edited.gz" ::: *ratio.gff.gz```
ShannonTown
CC BY-SA 4.0
2024-08-08T10:53:42.877Z
Huh, sorry @ShannonTown that looks like a bug in the markdown rendering of the site. I only have one backslash in the answer (you can see if you try to edit it) but for some reason it is rendered as three!
terdon
CC BY-SA 4.0
2024-08-07T12:23:24.743Z
2024-08-08T17:38:51.293Z
2024-08-08T17:38:51.293Z
terdon
<p>I have multiple gff files that I want to edit the start and end by 5 bp in batch. I found a relevant answer <a href="https://bioinformatics.stackexchange.com/questions/17715/edit-end-column-in-gff3-file">here</a> for single file, but I could not add comment there. The awk line works for my files if processing only one file, but when I try to do the same thing in batch, it returns syntax error. How should I do it with parallel?</p>
<pre><code>&gt; ls ./*ratio.gff.gz | parallel &quot;zcat {} | head | awk -F'\t' -v OFS='\t' '/^[^#]/{<span class="math-container">$5-=5}{$</span>4+=5}1' &quot;
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=


&gt; zcat xxxx_ratio.gff.gz | head | awk -F'\t' -v OFS='\t' '/^[^#]/{<span class="math-container">$5-=5}{$</span>4+=5}1'
##gff-version   3               5
# biweight_mean=-0.22621330627900907                    5
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5136    5179    0.60    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005131;count=2
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5181    5229    0.65    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005176;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5241    5287    0.41    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005236;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5286    5327    0.09    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005281;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5406    5452    0.92    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005401;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5466    5519    0.98    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005461;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5506    5559    0.26    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005501;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5576    5628    0.41    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005571;count=1
</code></pre>
<p>Thank you!</p>

CC BY-SA 4.0
2024-08-06T23:48:54.713Z
what happens if you move the awk command to a shell script? I suspect that the quoting is messing it up somehow.
Maximilian Press
CC BY-SA 4.0
2024-08-07T12:02:33.240Z
[_Never_ do `ls | command`](https://mywiki.wooledge.org/ParsingLs), that is asking for trouble. To be fair, it isn't causing issues here, but it is still a bad habit.
terdon
<awk><parallel>
Edit column in multiple GFF3 file using parallel and awk?
# Edit column in multiple GFF3 file using parallel and awk?

(Asked by: username_0 on 2024-08-06)

I have multiple gff files that I want to edit the start and end by 5 bp in batch. I found a relevant answer here for single file, but I could not add comment there. The awk line works for my files if processing only one file, but when I try to do the same thing in batch, it returns syntax error. How should I do it with parallel?
> ls ./*ratio.gff.gz | parallel "zcat {} | head | awk -F'\t' -v OFS='\t' '/^[^#]/{$5-=5}{$4+=5}1' "
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=
awk: line 1: syntax error at or near -=
awk: line 1: syntax error at or near +=


> zcat xxxx_ratio.gff.gz | head | awk -F'\t' -v OFS='\t' '/^[^#]/{$5-=5}{$4+=5}1'
##gff-version   3               5
# biweight_mean=-0.22621330627900907                    5
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5136    5179    0.60    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005131;count=2
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5181    5229    0.65    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005176;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5241    5287    0.41    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005236;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5286    5327    0.09    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005281;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5406    5452    0.92    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005401;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5466    5519    0.98    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005461;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5506    5559    0.26    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005501;count=1
chr2L   NimbleScan      31945102:MA_larvae_Exp1/MA_larvae_Ctrl1:BLOCK1  5576    5628    0.41    .       .       seq_id=chr2L:1-23011544;probe_id=CHR02LFS000005571;count=1

Thank you!

**Tags:** awk, parallel
> **Comments:**
> **username_1**: what happens if you move the awk command to a shell script? I suspect that the quoting is messing it up somehow.  
> **username_2**: [_Never_ do `ls | command`](https://mywiki.wooledge.org/ParsingLs), that is asking for trouble. To be fair, it isn't causing issues here, but it is still a bad habit.  

---

## 1 answers

### Answer by username_2 on 2024-08-07. Score: 2 | Accepted answer

You are getting a syntax error because your command is in double quotes. This means that the awk variables you are using, $4 and $5, are being expanded by the shell before launching awk, and since they are not defined, they are passed to awk as empty, making your command:
awk -F'\t' -v OFS='\t' '/^[^#]/{-=5}{+=5}1' "

And that is what gives you the syntax error. If you are 100% sure your file names contain no whitespace and no globbing characters, so you can use ls as you have done (which is generally a very bad idea), then this should work:
 ls ./*ratio.gff.gz | 
   parallel "zcat {} | head | awk -F'\t' -v OFS='\t' '/^[^#]/{\\\$5-=5}{\\\$4+=5}1' "

However, that won't change the original files and it isn't saving the output anywhere, so I doubt it's what you really want to do. This command will edit each file, and save the result into a new file with the same name plus edited.gz. So if you start with foo.gff.gz, you will get a new file called foo.gff.gz.edited.gz:
parallel "awk -F'\t' -v OFS='\t' '/^[^#]/{\\\$5-=5}{\\\$4+=5}1' <(zcat {}) | gzip > {}.edited.gz" ::: *gff.gz

Once you are sure this works as you want, you can rename all the edited.gz files back to the original name, overwriting the originals:
for f in *edited.gz; do mv -- "$f" "${f//.edited.gz}"; done


IMPORTANT: The solution above is affected by a markdown rendering bug. There should only ever be one \, as can be seen in the source of this answer, and not three. Make sure to replace all occurrences of \\\ in this answer with \.

> **Comments:**
> **username_0**: Thank you so much, this is exactly what I want to do. Somehow on the server I use (Debian) "\\\" also returns an error ```awk: 1: unexpected character '\' ```, if changed to "\", it worked.   ```parallel "awk -F'\t' -v OFS='\t' '/^[^#]/{\$5-=5}{\$4+=5}1' <(zcat {}) | gzip > {}.edited.gz" ::: *ratio.gff.gz```  
> **username_2**: Huh, sorry @ShannonTown that looks like a bug in the markdown rendering of the site. I only have one backslash in the answer (you can see if you try to edit it) but for some reason it is rendered as three!  

---

