<p>This is not the thing I do these days but ...</p>
<pre><code>gunzip -c file.fastq.gz | sed &quot;45500973,45500974d&quot; &gt; new.fastq
</code></pre>
<p>You were using <code>gzip</code> instead of <code>gunzip</code>, and that the source of the error. Finally the out file isn't compressed, so you probably want:</p>
<pre><code>gunzip -c file.fastq.gz | sed &quot;45500973,45500974d&quot; | gzip &gt; new.fastq.gz
</code></pre>
<p>That should work (I tested it and yes it works).</p>

CC BY-SA 4.0
2023-05-23T00:26:52.767Z
2023-05-23T11:17:34.113Z
2023-05-23T11:17:34.113Z
M__
<p>A <code>fastq</code> file has four line-separated fields per sequence. If you suspect that a sequence is corrupted, then you would want to delete all four lines related to that sequence. If you just delete the first two lines (<code>sequence identifier</code> and <code>raw sequence letters</code>), the file will most probably throw an error down the road. In your case, the sequence would look something like this -</p>
<pre><code>@VH00362:5:AAAV5KTHV:1:1210:64850:6869
GATTTGGGGTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF&gt;&gt;&gt;&gt;&gt;&gt;CCCCCCC65
</code></pre>
<p>So, the code to delete these four lines from a <code>fastq.gz</code> file would be -</p>
<pre><code>gunzip -c file.fastq.gz | sed &quot;45500973,45500976d&quot; | gzip &gt; new.fastq.gz
</code></pre>
<p>This <code>sed</code> command specifies a range of lines and says that delete lines from line number <code>45500973</code> to <code>45500976</code>.</p>

CC BY-SA 4.0
2023-05-24T15:34:27.357Z
2023-05-24T15:34:27.357Z
B. Kapoor
<p>Hi guys =) I'm sorry if this is a repeat but I haven't been able to find an answer using the search field or google.</p>
<p>I am trying to edit a fastq file that has a corrupt line. The read in question is VH00362:5:AAAV5KTHV:1:1210:64850:6869 where the read is just a massive repeat – not the expected 110bp. I am trying to remove two lines from this file and asking around  I've been told to use <code>sed</code> as in</p>
<p><code>gzip -c file.fastq.gz | sed &quot;45500973,45500974d&quot; &gt; new.fastq.gz</code></p>
<p>but when I run <code>wc -l </code> to check the number of lines the results are incoherent, as if it has deleted much more than the two lines I asked. Another issue that I'm seeing with this file is that every time I run <code>gzip</code> I get a <code>invalid compressed data--format violated</code> error, which tells me the file is corrupted but I'm doing my best to salvage whatever I can from the file as it is an irreplaceable sample.</p>
<p>Is there another way to simply delete these two lines? Any tool or command you could recommend? Any help is deeply appreciated.</p>

CC BY-SA 4.0
2023-05-23T11:19:14.067Z
Does `gzip -k file.fastq.gz` report an error? If so, I would first try to `gunzip` the file, or `zcat file.fastq.gz > file.fastq` and then operate on the decompressed one. I am not sure, but I suspect that a corrupted archive might not `cat` as expected and I would feel safer getting as much of the file decompressed as I can first. I am not even sure you will get anything, please try and report back.
terdon
<rna-seq><data-preprocessing>
How can I delete these lines in my fastq file?
# How can I delete these lines in my fastq file?

(Asked by: username_0 on 2023-05-23)

Hi guys =) I'm sorry if this is a repeat but I haven't been able to find an answer using the search field or google.

I am trying to edit a fastq file that has a corrupt line. The read in question is VH00362:5:AAAV5KTHV:1:1210:64850:6869 where the read is just a massive repeat – not the expected 110bp. I am trying to remove two lines from this file and asking around I've been told to use `sed` as in

`gzip -c file.fastq.gz | sed "45500973,45500974d" > new.fastq.gz`

but when I run `wc -l`  to check the number of lines the results are incoherent, as if it has deleted much more than the two lines I asked. Another issue that I'm seeing with this file is that every time I run `gzip` I get a `invalid compressed data--format violated` error, which tells me the file is corrupted but I'm doing my best to salvage whatever I can from the file as it is an irreplaceable sample.

Is there another way to simply delete these two lines? Any tool or command you could recommend? Any help is deeply appreciated.

**Tags:** rna-seq, data-preprocessing
> **Comments:**
> **username_1**: Does `gzip -k file.fastq.gz` report an error? If so, I would first try to `gunzip` the file, or `zcat file.fastq.gz > file.fastq` and then operate on the decompressed one. I am not sure, but I suspect that a corrupted archive might not `cat` as expected and I would feel safer getting as much of the file decompressed as I can first. I am not even sure you will get anything, please try and report back.  

---

## 2 answers

### Answer by username_2 on 2023-05-23. Score: 2

This is not the thing I do these days but ...
gunzip -c file.fastq.gz | sed "45500973,45500974d" > new.fastq

You were using gzip instead of gunzip, and that the source of the error. Finally the out file isn't compressed, so you probably want:
gunzip -c file.fastq.gz | sed "45500973,45500974d" | gzip > new.fastq.gz

That should work (I tested it and yes it works).


---

### Answer by username_3 on 2023-05-24. Score: 1

A fastq file has four line-separated fields per sequence. If you suspect that a sequence is corrupted, then you would want to delete all four lines related to that sequence. If you just delete the first two lines (sequence identifier and raw sequence letters), the file will most probably throw an error down the road. In your case, the sequence would look something like this -
@VH00362:5:AAAV5KTHV:1:1210:64850:6869
GATTTGGGGTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTGTTCAACTCACAGTTT
+
!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65

So, the code to delete these four lines from a fastq.gz file would be -
gunzip -c file.fastq.gz | sed "45500973,45500976d" | gzip > new.fastq.gz

This sed command specifies a range of lines and says that delete lines from line number 45500973 to 45500976.


---

