<p>The issue was caused by the corrupted input <code>.fastq</code> files that were damaged somehow upon uploading them from the local machine. We figured it out by <code>md5sum</code> command output comparison. We have not yet managed to actually run all of the script still, it is failing, but for another reason now. </p>

CC BY-SA 3.0
2017-12-14T22:39:22.913Z
2017-12-14T22:39:22.913Z
Nikita Vlasenko
<p>I am trying to run salmon and it keeps giving me 2 java exceptions:</p>

<pre><code>java.io.EOFException
    at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:268)
    at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258)
    at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164)
    at java.util.zip.GZIPInputStream.&lt;init&gt;(GZIPInputStream.java:79)
    at java.util.zip.GZIPInputStream.&lt;init&gt;(GZIPInputStream.java:91)
    at uk.ac.babraham.FastQC.Utilities.MultiMemberGZIPInputStream.&lt;init&gt;(MultiMemberGZIPInputStream.java:37)
</code></pre>

<p>and </p>

<pre><code>java.lang.ArrayIndexOutOfBoundsException: -1
    at uk.ac.babraham.FastQC.Modules.SequenceLengthDistribution.calculateDistribution(SequenceLengthDistribution.java:100)
    at uk.ac.babraham.FastQC.Modules.SequenceLengthDistribution.raisesError(SequenceLengthDistribution.java:184)
    at uk.ac.babraham.FastQC.Report.HTMLReportArchive.startDocument(HTMLReportArchive.java:336)
    at uk.ac.babraham.FastQC.Report.HTMLReportArchive.&lt;init&gt;(HTMLReportArchive.java:84)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.analysisComplete(OfflineRunner.java:155)
    at uk.ac.babraham.FastQC.Analysis.AnalysisRunner.run(AnalysisRunner.java:110)
    at java.lang.Thread.run(Thread.java:745)

    at uk.ac.babraham.FastQC.Sequence.FastQFile.&lt;init&gt;(FastQFile.java:80)
    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:106)
    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:62)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.processFile(OfflineRunner.java:129)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.&lt;init&gt;(OfflineRunner.java:102)
    at uk.ac.babraham.FastQC.FastQCApplication.main(FastQCApplication.java:316)
</code></pre>

<p>I installed <code>salmon</code> using <code>Anaconda</code>s' <code>conda install -c bioconda salmon</code> and all other necessary packages in the same way. I thought that it could be a problem with <code>fastqc</code>, so I uninstalled it and then installed it manually (through the fastqc<code>.zip</code> file), but the output remained the same.
I am using <code>Java8</code>:</p>

<pre><code>java -version
openjdk version "1.8.0_92"
OpenJDK Runtime Environment (Zulu 8.15.0.1-linux64) (build 1.8.0_92-b15)
OpenJDK 64-Bit Server VM (Zulu 8.15.0.1-linux64) (build 25.92-b15, mixed mode)
</code></pre>

<p>I am running the script on a cluster with <code>SLURM</code>. Locally on my laptop the script is running without any issues. I am really stuck and do not know what to try, so any suggestions would be greatly appreciated.</p>

<p><code>Update</code>
Another issue that I just found is the following (the last command that <code>salmon</code> is trying to execute):</p>

<pre><code>Exception : [Failed to read 8 bytes from input stream! Read 0]
salmon quant was invoked improperly.
</code></pre>

<p>Then I searched online for the issue and found the following thread:</p>

<p><a href="https://github.com/COMBINE-lab/salmon/issues/129" rel="nofollow noreferrer">https://github.com/COMBINE-lab/salmon/issues/129</a></p>

<p>However, updating <code>salmon</code> to the newest <code>0.9.1</code> version did not solve the issue.</p>

<p><code>Edit</code>
The command I am running is the following one:</p>

<pre><code>python "$path/salmon_rna_seq.py" -L 'IU' -E "$path/jyvo_experiment-
metadata.yaml" -T 'gencode_mouse_m13' -C "$path/salmon_rna_seq.yaml" -I 
"$path_to_fastq/$name-lec_1_r1.fq.gz" "$path_to_fastq/$name-
lec_1_r2.fq.gz" -O "$path/res/$name-lec_1" -S "$name-lec_1" -P '6' -X 'True'
</code></pre>

<p>The command is run by <code>Pypiper</code>: <a href="https://github.com/epigen/pypiper" rel="nofollow noreferrer">https://github.com/epigen/pypiper</a>
Inside of <code>salmon_rna_seq.py</code> I am also running commands to treat the data, the <code>salmon</code> part looks the following way:</p>

<pre><code>def salmon(self):
    cmd = "salmon quant -i " + self.indexed_transcriptome
    cmd += " -l " + self.lib_type

    if self.trim:
        cmd += " -1 " + self.sample_files['r1']['fastq']['trimmed_paired']
        cmd += " -2 " + self.sample_files['r2']['fastq']['trimmed_paired']
    else:
        cmd += " -1 " + self.sample_files['r1']['fastq']['chastity']
        cmd += " -2 " + self.sample_files['r2']['fastq']['chastity']

    cmd += " -o " + self.sample_dirs['salmon']['base']
    cmd += " --numBootstraps=30"
self.pipe_manager.run(cmd, self.sample_files['salmon'])
</code></pre>

<p>Commands that the <code>Pypiper</code> runs are the following ones (<code>_commands.sh</code> <code>Pypiper</code> file): </p>

<pre><code>zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz | grep -A 3 
'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c &gt; $PATH/res/old-lec_1/fastq
/chastity/old-lec_1_r1.fq.gz

zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz | grep -A 3 
'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c &gt; $PATH/res/old-lec_1/fastq
/chastity/old-lec_1_r2.fq.gz

java -jar $HOME/anaconda2/share/trimmomatic-0.36-5
/trimmomatic.jar PE -threads 6 -phred33 $PATH/res/old-lec_1/fastq/chastity/old-
lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz $PATH/res/old-
lec_1/fastq/trimmed/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/unpaired
/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz $PATH/res/old-
lec_1/fastq/trimmed/unpaired/old-lec_1_r2.fq.gz ILLUMINACLIP:$HOME/anaconda2/share/trimmomatic-0.36-5/adapters
/Small_List_Of_Adapters.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15
MINLEN:36

fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/merged

fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/chastity

fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/merged

fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/chastity

fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/trimmed

fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/trimmed

salmon quant -i $HOME/Genomes/mouse/salmon
/gencode.vM13.transcripts.fa.sidx -l IU -1 $PATH/res/old-lec_1/fastq/trimmed
/old-lec_1_r1.fq.gz -2 $PATH/res/old-lec_1/fastq/trimmed/old-
lec_1_r2.fq.gz -o $PATH/res/old-lec_1/quantification --numBootstraps=30
</code></pre>

<p><code>Edit</code></p>

<p>After running manually <code>gzip -t</code> on a file:</p>

<pre><code>-bash-4.2$ gzip -t old-lec_1_r1.fq.gz 
gzip: old-lec_1_r1.fq.gz: unexpected end of file
</code></pre>

<p>So, I guess the file is corrupted and the issue is with some library that is generating the file. However, reinstalling manually other libraries do not help.</p>

CC BY-SA 3.0
2017-12-12T23:57:55.197Z
Can you run `gzip -t` on the file(s) to see if it/they are corrupt? That'd be the interpretation of the error messages you're seeing. Make sure to run this on the cluster, in case there's some sort of "modified in transit" networking error going on.
Devon Ryan
CC BY-SA 3.0
2017-12-13T03:53:10.907Z
This would be my guess.  Also, however, I'm not sure how Java is involved (since Salmon is written completely in C++).
nomad
CC BY-SA 3.0
2017-12-13T05:06:14.143Z
It looks like FastQC is being invoked somewhere. Providing the precise commands invoked will help us troubleshoot the problem.
Daniel Standage
CC BY-SA 3.0
2017-12-13T09:59:00.550Z
Yes, please [edit] your question and show the exact commands you are using. It's hard to debug something if we don't know how to replicate it.
terdon
CC BY-SA 3.0
2017-12-13T10:46:12.103Z
Salmon is a red herring. The error message is clearly unrelated to Salmon.
Konrad Rudolph
CC BY-SA 3.0
2017-12-13T11:11:39.013Z
Oh god. I swear I didn’t notice the pun in my previous comment.
Konrad Rudolph
CC BY-SA 3.0
2017-12-13T14:06:45.277Z
+1 Konrad, for the unintentional pun ;P.
nomad
CC BY-SA 3.0
2017-12-13T18:24:32.033Z
Where does old-lec_1_r1.fq.gz come from?  That is the issue.
Bioathlete
CC BY-SA 3.0
2017-12-13T19:04:58.490Z
FastQC is involved in generating it mainly
Nikita Vlasenko
CC BY-SA 3.0
2017-12-14T08:05:49.253Z
FastQC has absolutely nothing to do with generating the file, it just produces some QC metrics. It's unclear (A) why you're running grep on the files and (B) which of the multiple copies of old-lec_1_r1.fq.gz is the first that's corrupt.
Devon Ryan
<rna-seq><quantification><salmon>
Exception : [Failed to read 8 bytes from input stream! Read 0] salmon quant was invoked improperly
# Exception : [Failed to read 8 bytes from input stream! Read 0] salmon quant was invoked improperly

(Asked by: username_0 on 2017-12-12)

I am trying to run salmon and it keeps giving me 2 java exceptions:
java.io.EOFException
    at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:268)
    at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258)
    at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164)
    at java.util.zip.GZIPInputStream.(GZIPInputStream.java:79)
    at java.util.zip.GZIPInputStream.(GZIPInputStream.java:91)
    at uk.ac.babraham.FastQC.Utilities.MultiMemberGZIPInputStream.(MultiMemberGZIPInputStream.java:37)

and 
java.lang.ArrayIndexOutOfBoundsException: -1
    at uk.ac.babraham.FastQC.Modules.SequenceLengthDistribution.calculateDistribution(SequenceLengthDistribution.java:100)
    at uk.ac.babraham.FastQC.Modules.SequenceLengthDistribution.raisesError(SequenceLengthDistribution.java:184)
    at uk.ac.babraham.FastQC.Report.HTMLReportArchive.startDocument(HTMLReportArchive.java:336)
    at uk.ac.babraham.FastQC.Report.HTMLReportArchive.(HTMLReportArchive.java:84)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.analysisComplete(OfflineRunner.java:155)
    at uk.ac.babraham.FastQC.Analysis.AnalysisRunner.run(AnalysisRunner.java:110)
    at java.lang.Thread.run(Thread.java:745)

    at uk.ac.babraham.FastQC.Sequence.FastQFile.(FastQFile.java:80)
    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:106)
    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:62)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.processFile(OfflineRunner.java:129)
    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.(OfflineRunner.java:102)
    at uk.ac.babraham.FastQC.FastQCApplication.main(FastQCApplication.java:316)

I installed salmon using Anacondas' conda install -c bioconda salmon and all other necessary packages in the same way. I thought that it could be a problem with fastqc, so I uninstalled it and then installed it manually (through the fastqc.zip file), but the output remained the same.
I am using Java8:
java -version
openjdk version "1.8.0_92"
OpenJDK Runtime Environment (Zulu 8.15.0.1-linux64) (build 1.8.0_92-b15)
OpenJDK 64-Bit Server VM (Zulu 8.15.0.1-linux64) (build 25.92-b15, mixed mode)

I am running the script on a cluster with SLURM. Locally on my laptop the script is running without any issues. I am really stuck and do not know what to try, so any suggestions would be greatly appreciated.
Update
Another issue that I just found is the following (the last command that salmon is trying to execute):
Exception : [Failed to read 8 bytes from input stream! Read 0]
salmon quant was invoked improperly.

Then I searched online for the issue and found the following thread:
https://github.com/COMBINE-lab/salmon/issues/129
However, updating salmon to the newest 0.9.1 version did not solve the issue.
Edit
The command I am running is the following one:
python "$path/salmon_rna_seq.py" -L 'IU' -E "$path/jyvo_experiment-
metadata.yaml" -T 'gencode_mouse_m13' -C "$path/salmon_rna_seq.yaml" -I 
"$path_to_fastq/$name-lec_1_r1.fq.gz" "$path_to_fastq/$name-
lec_1_r2.fq.gz" -O "$path/res/$name-lec_1" -S "$name-lec_1" -P '6' -X 'True'

The command is run by Pypiper: https://github.com/epigen/pypiper
Inside of salmon_rna_seq.py I am also running commands to treat the data, the salmon part looks the following way:
def salmon(self):
    cmd = "salmon quant -i " + self.indexed_transcriptome
    cmd += " -l " + self.lib_type

    if self.trim:
        cmd += " -1 " + self.sample_files['r1']['fastq']['trimmed_paired']
        cmd += " -2 " + self.sample_files['r2']['fastq']['trimmed_paired']
    else:
        cmd += " -1 " + self.sample_files['r1']['fastq']['chastity']
        cmd += " -2 " + self.sample_files['r2']['fastq']['chastity']

    cmd += " -o " + self.sample_dirs['salmon']['base']
    cmd += " --numBootstraps=30"
self.pipe_manager.run(cmd, self.sample_files['salmon'])

Commands that the Pypiper runs are the following ones (_commands.sh Pypiper file): 
zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz | grep -A 3 
'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c > $PATH/res/old-lec_1/fastq
/chastity/old-lec_1_r1.fq.gz

zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz | grep -A 3 
'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c > $PATH/res/old-lec_1/fastq
/chastity/old-lec_1_r2.fq.gz

java -jar $HOME/anaconda2/share/trimmomatic-0.36-5
/trimmomatic.jar PE -threads 6 -phred33 $PATH/res/old-lec_1/fastq/chastity/old-
lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz $PATH/res/old-
lec_1/fastq/trimmed/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/unpaired
/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz $PATH/res/old-
lec_1/fastq/trimmed/unpaired/old-lec_1_r2.fq.gz ILLUMINACLIP:$HOME/anaconda2/share/trimmomatic-0.36-5/adapters
/Small_List_Of_Adapters.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15
MINLEN:36

fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/merged

fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/chastity

fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/merged

fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/chastity

fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/trimmed

fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/trimmed

salmon quant -i $HOME/Genomes/mouse/salmon
/gencode.vM13.transcripts.fa.sidx -l IU -1 $PATH/res/old-lec_1/fastq/trimmed
/old-lec_1_r1.fq.gz -2 $PATH/res/old-lec_1/fastq/trimmed/old-
lec_1_r2.fq.gz -o $PATH/res/old-lec_1/quantification --numBootstraps=30

Edit
After running manually gzip -t on a file:
-bash-4.2$ gzip -t old-lec_1_r1.fq.gz 
gzip: old-lec_1_r1.fq.gz: unexpected end of file

So, I guess the file is corrupted and the issue is with some library that is generating the file. However, reinstalling manually other libraries do not help.

**Tags:** rna-seq, quantification, salmon
> **Comments:**
> **username_1**: Can you run `gzip -t` on the file(s) to see if it/they are corrupt? That'd be the interpretation of the error messages you're seeing. Make sure to run this on the cluster, in case there's some sort of "modified in transit" networking error going on.  
> **username_2**: This would be my guess.  Also, however, I'm not sure how Java is involved (since Salmon is written completely in C++).  
> **username_3**: It looks like FastQC is being invoked somewhere. Providing the precise commands invoked will help us troubleshoot the problem.  
> **username_4**: Yes, please [edit] your question and show the exact commands you are using. It's hard to debug something if we don't know how to replicate it.  
> **username_5**: Salmon is a red herring. The error message is clearly unrelated to Salmon.  
> **username_5**: Oh god. I swear I didn’t notice the pun in my previous comment.  
> **username_2**: +1 Konrad, for the unintentional pun ;P.  
> **username_6**: Where does old-lec_1_r1.fq.gz come from?  That is the issue.  
> **username_0**: FastQC is involved in generating it mainly  
> **username_1**: FastQC has absolutely nothing to do with generating the file, it just produces some QC metrics. It's unclear (A) why you're running grep on the files and (B) which of the multiple copies of old-lec_1_r1.fq.gz is the first that's corrupt.  

---

## 1 answers

### Answer by username_0 on 2017-12-14. Score: 1 | Accepted answer

The issue was caused by the corrupted input `.fastq` files that were damaged somehow upon uploading them from the local machine. We figured it out by `md5sum` command output comparison. We have not yet managed to actually run all of the script still, it is failing, but for another reason now.


---

