<p>I am trying to process fastq files in order to build gene co-expression networks (by following <a href="https://www.ncbi.nlm.nih.gov/pubmed/29525965" rel="nofollow noreferrer">this tutorial</a>). When I download any fastq file from ENA, and try to process it with the command:
<code>ls *.fastq.gz | xargs -n1 fastqc $1</code>, I get:</p>
<pre><code>Failed to process SRR494099.fastq.gz
java.io.EOFException
    at java.base/java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:269)
    at java.base/java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:259)
    at java.base/java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:165)
    at java.base/java.util.zip.GZIPInputStream.&lt;init&gt;(GZIPInputStream.java:80)
    at java.base/java.util.zip.GZIPInputStream.&lt;init&gt;(GZIPInputStream.java:92)
    at uk.ac.babraham.FastQC.Utilities.MultiMemberGZIPInputStream.&lt;init&gt;(MultiMemberGZIPInputStream.java:37)
    at uk.ac.babraham.FastQC.Sequence.FastQFile.&lt;init&gt;(FastQFile.java:80)
    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:106)
    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:62)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.processFile(OfflineRunner.java:159)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.&lt;init&gt;(OfflineRunner.java:121)
    at uk.ac.babraham.FastQC.FastQCApplication.main(FastQCApplication.java:316)

</code></pre>
<p>Similarly, when I just download the fastq files and try to decompress them using gunzip, I get, for example: <code>gzip: SRR494099.fastq.gz: unexpected end of file</code>
Why is this happening?</p>

CC BY-SA 4.0
2020-05-17T16:11:33.417Z
Likely that your fastq file is not successfully download based on `unexpected end of file`. Try `zcat SRR494099.fastq.gz | tail`
Phoenix Mu
CC BY-SA 4.0
2020-05-17T16:13:28.163Z
@PhoenixMu I get `gzip: SRR494099.fastq.gz: unexpected end of file` when I do this
An Ignorant Wanderer
CC BY-SA 4.0
2020-05-17T18:13:54.613Z
@ATpoint yeah I was just trying to see if the problem was with fastqc or with the file itself. But none of the files that I download from ENA are working
An Ignorant Wanderer
CC BY-SA 4.0
2020-05-17T21:35:51.257Z
do they provide file hashes such as e.g. checksums that you can use to ensure that nothing weird happened to the files?
Maximilian Press
CC BY-SA 4.0
2020-05-17T22:17:45.580Z
@MaximilianPress no but it doesn't just happen with the files I'm interested in. Any fastq file.
An Ignorant Wanderer
CC BY-SA 4.0
2020-05-18T17:31:49.147Z
Hm, that is difficult. If you can't verify the integrity of your files it will be hard to say what the issue is- but it's almost certain that they are messed up somehow. I'd follow ATpoint's suggestion or other alternative means of downloading. Possibly also switching to a clean environment with robust networking such as a fresh node on a HPC cluster might clear something up with your downloads.
Maximilian Press
<fastq><gene-expression>
Unable to gunzip fastq files from ENA
# Unable to gunzip fastq files from ENA

(Asked by: username_0 on 2020-05-17)

I am trying to process fastq files in order to build gene co-expression networks (by following this tutorial). When I download any fastq file from ENA, and try to process it with the command:
ls *.fastq.gz | xargs -n1 fastqc $1, I get:
Failed to process SRR494099.fastq.gz
java.io.EOFException
    at java.base/java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:269)
    at java.base/java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:259)
    at java.base/java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:165)
    at java.base/java.util.zip.GZIPInputStream.(GZIPInputStream.java:80)
    at java.base/java.util.zip.GZIPInputStream.(GZIPInputStream.java:92)
    at uk.ac.babraham.FastQC.Utilities.MultiMemberGZIPInputStream.(MultiMemberGZIPInputStream.java:37)
    at uk.ac.babraham.FastQC.Sequence.FastQFile.(FastQFile.java:80)
    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:106)
    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:62)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.processFile(OfflineRunner.java:159)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.(OfflineRunner.java:121)
    at uk.ac.babraham.FastQC.FastQCApplication.main(FastQCApplication.java:316)


Similarly, when I just download the fastq files and try to decompress them using gunzip, I get, for example: gzip: SRR494099.fastq.gz: unexpected end of file
Why is this happening?

**Tags:** fastq, gene-expression
> **Comments:**
> **username_1**: Likely that your fastq file is not successfully download based on `unexpected end of file`. Try `zcat SRR494099.fastq.gz | tail`  
> **username_0**: @PhoenixMu I get `gzip: SRR494099.fastq.gz: unexpected end of file` when I do this  
> **username_0**: @ATpoint yeah I was just trying to see if the problem was with fastqc or with the file itself. But none of the files that I download from ENA are working  
> **username_2**: do they provide file hashes such as e.g. checksums that you can use to ensure that nothing weird happened to the files?  
> **username_0**: @MaximilianPress no but it doesn't just happen with the files I'm interested in. Any fastq file.  
> **username_2**: Hm, that is difficult. If you can't verify the integrity of your files it will be hard to say what the issue is- but it's almost certain that they are messed up somehow. I'd follow ATpoint's suggestion or other alternative means of downloading. Possibly also switching to a clean environment with robust networking such as a fresh node on a HPC cluster might clear something up with your downloads.  

---

## 0 answers

