<p>First of all <em>never</em> do <code>for foo in $(find ...)</code>. That is very bad practice as it can't handle even slightly odd file names like those with a space. See <a href="https://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_.2A.mp3.29" rel="nofollow noreferrer">bash pitfall #1</a>.</p>
<p>Next, you don't need <code>pwd</code>, just run <code>find -type f -name &quot;*.fa&quot;</code> which will search in the current directory by default and unlike <code>find `pwd`... </code>, won't break if your path contains whitespace.</p>
<p>That's just general best practice, the real issue here is that you are probably running the script like this:</p>
<pre><code>sh /path/to/script.sh
</code></pre>
<p>Right? And you are using a Debian-based system like Ubuntu, right? On such systems, <code>/bin/sh</code> is actually not bash but a very basic POSIX compatible shell called <code>dash</code> and <code>dash</code> doesn't understand the <code>(( i++%waitevery==0 ))</code> syntax.</p>
<p>Since you have a shebang line (<code>#!/bin/bash</code>), you can simply make your script executable (<code>chmod a+x /path/to/script.sh</code>) and then run the script directly like this:</p>
<pre><code>/path/to/script.sh
</code></pre>
<p>Or, alternatively, run it explicitly with <code>bash</code> instead of <code>sh</code>:</p>
<pre><code>bash /path/to/script.sh
</code></pre>
<p>Either should work and if your file names and paths are sane, the issues I mentioned above won't be relevant. But to keep things nice and clean, I would also change the script to (note that this assumes you are using Linux and GNU <code>find</code>):</p>
<pre class="lang-sh prettyprint-override"><code>#!/bin/bash
i=1
waitevery=30

mkdir -p out

find -type f -name &quot;*.fa&quot; -print0 |
  while IFS= read -r -d '' j; do 
    echo &quot;Iteration: <span class="math-container">$i; File: $</span>j&quot;
    filename=<span class="math-container">$(basename "$</span>j&quot;)
    python3 iprscan5.py \
        --goterms \
        --pathways \
        --email=&quot;blabla@protonmail.com&quot; \
        --outfile=out/&quot;<span class="math-container">$filename" \
    --outformat=tsv \
    --quiet \
    "$</span>j&quot; &amp; (( i++%waitevery==0 )) &amp;&amp; wait
done
</code></pre>

CC BY-SA 4.0
2022-08-17T20:02:37.503Z
I asked them how they're running 'submit.sh` for the reason you picked precisely - they're probably using `sh` and hitting a POSIX compatibility wall.
Ram RS
CC BY-SA 4.0
2022-08-17T11:08:10.003Z
2022-08-17T11:08:10.003Z
terdon
<p>I'd guess that the specific error that's coming up is because there are spaces missing from the wait line. This seems to be a problem in the source script, i.e.:</p>
<pre><code>$j &amp; (( i++%waitevery==0 )) &amp;&amp; wait
</code></pre>
<p>should probably be:</p>
<pre><code>$j &amp; (( i++ % waitevery == 0 )) &amp;&amp; wait
</code></pre>
<p>but for me (following @terdon's reproduction advice via <code>dash -c &quot;i=9; ((++i%5==0)) &amp;&amp; echo 'worked!'&quot;</code>) that only changes the thing that is not found from <code>i++%waitevery==0</code> to <code>i++</code>, suggesting that the post-increment operator is not supported in the shell. Post-increment <em>is</em> <a href="https://linuxhint.com/bash_operator_examples/#o13" rel="nofollow noreferrer">supported in bash</a>, suggesting that this script is being run (as @RamRS commented) in an unexpected way.</p>
<p>Try running the script directly:</p>
<pre><code>./interpro_batch_submit.sh
</code></pre>
<p>Or explicitly using bash:</p>
<pre><code>bash ./interpro_batch_submit.sh
</code></pre>
<p>... but see @terdon's answer for a better approach to doing this.</p>

CC BY-SA 4.0
2022-08-17T11:50:59.717Z
2022-08-17T21:55:08.610Z
2022-08-17T21:55:08.610Z
gringer
<p>hey i have the problem with bash script. It should run interproscan REST
Im green in bash scripting. I found this script on the <a href="https://gist.github.com/terrymun/889f28314643d429496881adbdd40039" rel="nofollow noreferrer">web</a>:</p>
<pre><code>#!/bin/bash
i=1
waitevery=30

mkdir -p out

for j in <span class="math-container">$(find `pwd` -type f -name "*.fa")
do
    echo "Iteration: $</span>i; File: <span class="math-container">$j"
    filename=$</span>(basename &quot;<span class="math-container">$j")
    python3 iprscan5.py \
        --goterms \
        --pathways \
        --email=blabla@protonmail.com \
        --outfile=out/$</span>{filename} \
        --outformat=tsv \
        --quiet \
        $j &amp; (( i++%waitevery==0 )) &amp;&amp; wait
done
</code></pre>
<p>out:</p>
<pre><code>Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtr.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xts.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtt.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtu.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtv.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtw.fa
</code></pre>
<p>Edit:</p>
<p>I think I forgot to add <code>--sequence</code>, but something was still wrong after it.</p>

CC BY-SA 4.0
2022-08-16T17:19:43.890Z
Also, I'm assuming you're saving the script as `submit.sh` and then running it. How are you running it exactly?
Ram RS
CC BY-SA 4.0
2022-08-17T11:15:17.800Z
@RamRS `wait` will wait until all background jobs have finished. `(( i++%waitevery==0 ))` will be true whenever the current value of `i` modulo `waitevery` is 0, so this is a really neat trick to make the script wait for all background processes to execute every N processes. Quite clever, that.
terdon
<bash>
bash script for interproscan REST
# bash script for interproscan REST

(Asked by: username_0 on 2022-08-16)

hey i have the problem with bash script. It should run interproscan REST
Im green in bash scripting. I found this script on the web:
#!/bin/bash
i=1
waitevery=30

mkdir -p out

for j in $(find `pwd` -type f -name "*.fa")
do
    echo "Iteration: $i; File: $j"
    filename=$(basename "$j")
    python3 iprscan5.py \
        --goterms \
        --pathways \
        --email=blabla@protonmail.com \
        --outfile=out/${filename} \
        --outformat=tsv \
        --quiet \
        $j & (( i++%waitevery==0 )) && wait
done

out:
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtr.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xts.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtt.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtu.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtv.fa
interpro_batch_submit.sh: 18: i++%waitevery==0: not found
Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtw.fa

Edit:
I think I forgot to add --sequence, but something was still wrong after it.

**Tags:** bash
> **Comments:**
> **username_1**: Also, I'm assuming you're saving the script as `submit.sh` and then running it. How are you running it exactly?  
> **username_2**: @RamRS `wait` will wait until all background jobs have finished. `(( i++%waitevery==0 ))` will be true whenever the current value of `i` modulo `waitevery` is 0, so this is a really neat trick to make the script wait for all background processes to execute every N processes. Quite clever, that.  

---

## 2 answers

### Answer by username_2 on 2022-08-17. Score: 1 | Accepted answer

First of all never do for foo in $(find ...). That is very bad practice as it can't handle even slightly odd file names like those with a space. See bash pitfall #1.
Next, you don't need pwd, just run find -type f -name "*.fa" which will search in the current directory by default and unlike find `pwd`... , won't break if your path contains whitespace.
That's just general best practice, the real issue here is that you are probably running the script like this:
sh /path/to/script.sh

Right? And you are using a Debian-based system like Ubuntu, right? On such systems, /bin/sh is actually not bash but a very basic POSIX compatible shell called dash and dash doesn't understand the (( i++%waitevery==0 )) syntax.
Since you have a shebang line (#!/bin/bash), you can simply make your script executable (chmod a+x /path/to/script.sh) and then run the script directly like this:
/path/to/script.sh

Or, alternatively, run it explicitly with bash instead of sh:
bash /path/to/script.sh

Either should work and if your file names and paths are sane, the issues I mentioned above won't be relevant. But to keep things nice and clean, I would also change the script to (note that this assumes you are using Linux and GNU find):
#!/bin/bash
i=1
waitevery=30

mkdir -p out

find -type f -name "*.fa" -print0 |
  while IFS= read -r -d '' j; do 
    echo "Iteration: $i; File: $j"
    filename=$(basename "$j")
    python3 iprscan5.py \
        --goterms \
        --pathways \
        --email="blabla@protonmail.com" \
        --outfile=out/"$filename" \
    --outformat=tsv \
    --quiet \
    "$j" & (( i++%waitevery==0 )) && wait
done

> **Comments:**
> **username_1**: I asked them how they're running 'submit.sh` for the reason you picked precisely - they're probably using `sh` and hitting a POSIX compatibility wall.  

---

### Answer by username_3 on 2022-08-17. Score: 1

I'd guess that the specific error that's coming up is because there are spaces missing from the wait line. This seems to be a problem in the source script, i.e.:
$j & (( i++%waitevery==0 )) && wait

should probably be:
$j & (( i++ % waitevery == 0 )) && wait

but for me (following @terdon's reproduction advice via dash -c "i=9; ((++i%5==0)) && echo 'worked!'") that only changes the thing that is not found from i++%waitevery==0 to i++, suggesting that the post-increment operator is not supported in the shell. Post-increment is supported in bash, suggesting that this script is being run (as @RamRS commented) in an unexpected way.
Try running the script directly:
./interpro_batch_submit.sh

Or explicitly using bash:
bash ./interpro_batch_submit.sh

... but see @terdon's answer for a better approach to doing this.


---

