<p>Snakemake is using bash strict mode to ensure best practice error reporting in shell commands. Unfortunately some tools violate bash strict mode.</p>

<p>More details of the error, <a href="https://snakemake.readthedocs.io/en/stable/project_info/faq.html#my-shell-command-fails-with-exit-code-0-from-within-a-pipe-what-s-wrong" rel="nofollow noreferrer">here</a>. Alternatively, you can use if you don't want disable the strict mode.</p>

<pre><code>zcat file.fastq.gz  | awk '(NR&lt;=400000)' &gt; subset.fastq
</code></pre>

CC BY-SA 4.0
2020-04-02T07:42:18.847Z
This solves my problem exactly (the awk solution), it is fast and does what my original command does but without pipefail error. Thank you.
Freek
CC BY-SA 4.0
2020-04-02T15:40:38.517Z
@Freek another option would be to add `|| true` to give a 0 exit status
Chris_Rands
CC BY-SA 4.0
2020-03-31T22:10:12.203Z
2020-03-31T22:10:12.203Z
zorbax
<p>All,</p>

<p>As the first step in my Snakemake pipeline, I generate small fastq files from all input fastq.gz files. This is to run a quick round of "infer_experiment.py" (from RSeQC, to get the kit strandedness). However, Snakemake says this step fails. I think I know why, it is because a command like this</p>

<pre><code>zcat fastq_raw/20190011-043_S43_R1_001.fastq.gz | head -n 400000 &gt; infer_strandedness/20190011-043_S43_subset_R1.fastq
</code></pre>

<p>Ends with an error: <code>gzip: stdout: Broken pipe</code> and Snakemake uses Bash strict mode. (Snakemake reports: <code>(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)</code>). So my question is:</p>

<p>How to avoid this, or, are there other truncating methods that do finish cleanly? Preferebly they are also efficient (seqtk offers subsampling but this parses the entire fastq file!).</p>

CC BY-SA 4.0
2020-03-31T17:23:12.997Z
You're better off using seqtk even if it's slower since you're then not performing a highly biased sampling from the beginning of the file, where the quality tends to be worse.
Devon Ryan
CC BY-SA 4.0
2020-04-02T07:19:35.240Z
But the library is randomized during prep and flowing into the chip, why would the first part of the fastq file (first scanned tile of the chip) be biased? A bit worse quality would not be a problem if I only want to detect the strandedness...
Freek
CC BY-SA 4.0
2020-04-02T07:20:57.900Z
The beginning of the files represent the edge of the flow cell where the quality is lower.
Devon Ryan
<snakemake>
truncate fastq.gz file without error in Snakemake
# truncate fastq.gz file without error in Snakemake

(Asked by: username_0 on 2020-03-31)

All,
As the first step in my Snakemake pipeline, I generate small fastq files from all input fastq.gz files. This is to run a quick round of "infer_experiment.py" (from RSeQC, to get the kit strandedness). However, Snakemake says this step fails. I think I know why, it is because a command like this
zcat fastq_raw/20190011-043_S43_R1_001.fastq.gz | head -n 400000 > infer_strandedness/20190011-043_S43_subset_R1.fastq

Ends with an error: gzip: stdout: Broken pipe and Snakemake uses Bash strict mode. (Snakemake reports: (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)). So my question is:
How to avoid this, or, are there other truncating methods that do finish cleanly? Preferebly they are also efficient (seqtk offers subsampling but this parses the entire fastq file!).

**Tags:** snakemake
> **Comments:**
> **username_1**: You're better off using seqtk even if it's slower since you're then not performing a highly biased sampling from the beginning of the file, where the quality tends to be worse.  
> **username_0**: But the library is randomized during prep and flowing into the chip, why would the first part of the fastq file (first scanned tile of the chip) be biased? A bit worse quality would not be a problem if I only want to detect the strandedness...  
> **username_1**: The beginning of the files represent the edge of the flow cell where the quality is lower.  

---

## 1 answers

### Answer by username_2 on 2020-03-31. Score: 1 | Accepted answer

Snakemake is using bash strict mode to ensure best practice error reporting in shell commands. Unfortunately some tools violate bash strict mode.
More details of the error, here. Alternatively, you can use if you don't want disable the strict mode.
zcat file.fastq.gz  | awk '(NR<=400000)' > subset.fastq

> **Comments:**
> **username_0**: This solves my problem exactly (the awk solution), it is fast and does what my original command does but without pipefail error. Thank you.  
> **username_3**: @Freek another option would be to add `|| true` to give a 0 exit status  

---

