<pre><code>[W::sam_read1] Parse error at line 1
samtools sort: truncated file. Aborting
</code></pre>

<p>SAMtools sort has been unable to parse its input, which it thought was SAM (mostly because it couldn't be recognised as another format e.g. BAM).</p>

<p>This is because <code>sed 's/^/LP1-/'</code> is putting <code>LP1-</code> at the front of every line. If the output of samtools fixmate is SAM, then this LP1 is garbling the SAM header lines. If the output of samtools fixmate is BAM (as it probably is), this LP1 is completely garbling the data.</p>

<p>What were you trying to achieve with this <code>sed</code> command?</p>

CC BY-SA 4.0
2019-04-08T23:05:48.530Z
It is attended that `sed 's/^/LP1-/' is putting LP1- at the front of every line because I would like to use [SGSautoSNP.py](https://www.mdpi.com/2079-7737/1/2/370/htm) to detect different cultivars. How is it possible to fix the above pipeline?
user977828
CC BY-SA 4.0
2019-04-08T13:56:39.323Z
2019-04-08T13:56:39.323Z
John Marshall
<p>I have the following alignment script which uses BWA:</p>

<pre><code>#!/bin/bash
#usage: sh align_pbs.sh /scratch/waterhouse_team/benth/dbg2olc-40x/good/ output.racon2.fasta bam-soap

mkdir $3

for r1 in <span class="math-container">$(find $</span>1 -name "*R1*.gz");
do
  base=<span class="math-container">${r1%_R1*}
  output=$</span>(basename <span class="math-container">$(echo $</span>r1 | sed 's/R1//g'))
  r2=<span class="math-container">$(echo $</span>r1 | sed 's/R1/R2/g') 

  #Get read group infomration:
  #Source: https://www.biostars.org/p/280837/#310132
  header=<span class="math-container">$(zcat $</span>r1 | head -n 1)
  id=<span class="math-container">$(echo $</span>header | head -n 1 | cut -f 1-4 -d":" | sed 's/@//' | sed 's/:/_/g')
  sm=<span class="math-container">$(echo $</span>header | head -n 1 | grep -Eo "[ATGCN]+<span class="math-container">$")
  #echo "Read Group @RG\tID:$</span>id\tSM:<span class="math-container">$id"_"$</span>sm\tLB:<span class="math-container">$id"_"$</span>sm\tPL:ILLUMINA"
  echo "@RG\tID:<span class="math-container">${output}\tSM:$</span>{output}\tLB:${output}\tPL:ILLUMINA"

  #cat &lt;&lt;EOF
  qsub &lt;&lt;EOF
#!/bin/bash -l

#PBS -N $output
#PBS -l walltime=48:00:00
#PBS -j oe
#PBS -l mem=70G
#PBS -l ncpus=12
#PBS -M m.lorenc@qut.edu.au
##PBS -m bea

cd \$PBS_O_WORKDIR

conda activate bwa-hic
bwa mem \
<span class="math-container">$2 $</span>r1 <span class="math-container">$r2 \
-t 12 \
-R "$</span>(echo "@RG\tID:<span class="math-container">${output}\tSM:$</span>{output}\tLB:<span class="math-container">${output}\tPL:ILLUMINA")" | samblaster -r | samtools view -@ 12 -h -f 0x2 -F 2316 - | samtools fixmate - - | sed 's/^/LP1-/' | samtools sort -@ 12 - -o $</span>{3}/${output}.sorted.dedup.bam

EOF

done
</code></pre>

<p>Unfortunatelye, this pipeline cuased the following erros:</p>

<pre><code>samblaster: Version 0.1.24
samblaster: Inputting from stdin
samblaster: Outputting to stdout
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 1200000 sequences (120000000 bp)...
[M::process] read 1200000 sequences (120000000 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (2, 316829, 6, 4)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (432, 474, 509)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (278, 663)
[M::mem_pestat] mean and std.dev: (471.59, 53.66)
[M::mem_pestat] low and high boundaries for proper pairs: (201, 740)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 1200000 reads in 615.287 CPU sec, 58.490 real sec
samblaster: Loaded 19 header sequence entries.
[W::sam_read1] Parse error at line 1
samtools sort: truncated file. Aborting
sed: couldn't write 567 items to stdout: Broken pipe
[E::bgzf_flush] File write failed (wrong size)
samtools fixmate: Couldn't write to output file: Broken pipe
[E::bgzf_close] File write failed
[bam_mating] error while closing output file
samtools view: writing to standard output failed: Broken pipe
samtools view: error closing standard output: -1
samblaster: Unable to write to output file.
</code></pre>

<p>Thank you in advance,</p>

<samtools><bwa>
samtools view: writing to standard output failed: Broken pipe
# samtools view: writing to standard output failed: Broken pipe

(Asked by: username_0 on 2019-04-08)

I have the following alignment script which uses BWA:
#!/bin/bash
#usage: sh align_pbs.sh /scratch/waterhouse_team/benth/dbg2olc-40x/good/ output.racon2.fasta bam-soap

mkdir $3

for r1 in $(find $1 -name "*R1*.gz");
do
  base=${r1%_R1*}
  output=$(basename $(echo $r1 | sed 's/R1//g'))
  r2=$(echo $r1 | sed 's/R1/R2/g') 

  #Get read group infomration:
  #Source: https://www.biostars.org/p/280837/#310132
  header=$(zcat $r1 | head -n 1)
  id=$(echo $header | head -n 1 | cut -f 1-4 -d":" | sed 's/@//' | sed 's/:/_/g')
  sm=$(echo $header | head -n 1 | grep -Eo "[ATGCN]+$")
  #echo "Read Group @RG\tID:$id\tSM:$id"_"$sm\tLB:$id"_"$sm\tPL:ILLUMINA"
  echo "@RG\tID:${output}\tSM:${output}\tLB:${output}\tPL:ILLUMINA"

  #cat <$2 $r1 $r2 \
-t 12 \
-R "$(echo "@RG\tID:${output}\tSM:${output}\tLB:${output}\tPL:ILLUMINA")" | samblaster -r | samtools view -@ 12 -h -f 0x2 -F 2316 - | samtools fixmate - - | sed 's/^/LP1-/' | samtools sort -@ 12 - -o ${3}/${output}.sorted.dedup.bam

EOF

done

Unfortunatelye, this pipeline cuased the following erros:
samblaster: Version 0.1.24
samblaster: Inputting from stdin
samblaster: Outputting to stdout
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 1200000 sequences (120000000 bp)...
[M::process] read 1200000 sequences (120000000 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (2, 316829, 6, 4)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (432, 474, 509)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (278, 663)
[M::mem_pestat] mean and std.dev: (471.59, 53.66)
[M::mem_pestat] low and high boundaries for proper pairs: (201, 740)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 1200000 reads in 615.287 CPU sec, 58.490 real sec
samblaster: Loaded 19 header sequence entries.
[W::sam_read1] Parse error at line 1
samtools sort: truncated file. Aborting
sed: couldn't write 567 items to stdout: Broken pipe
[E::bgzf_flush] File write failed (wrong size)
samtools fixmate: Couldn't write to output file: Broken pipe
[E::bgzf_close] File write failed
[bam_mating] error while closing output file
samtools view: writing to standard output failed: Broken pipe
samtools view: error closing standard output: -1
samblaster: Unable to write to output file.

Thank you in advance,

**Tags:** samtools, bwa

---

## 1 answers

### Answer by username_1 on 2019-04-08. Score: 3

[W::sam_read1] Parse error at line 1
samtools sort: truncated file. Aborting

SAMtools sort has been unable to parse its input, which it thought was SAM (mostly because it couldn't be recognised as another format e.g. BAM).
This is because sed 's/^/LP1-/' is putting LP1- at the front of every line. If the output of samtools fixmate is SAM, then this LP1 is garbling the SAM header lines. If the output of samtools fixmate is BAM (as it probably is), this LP1 is completely garbling the data.
What were you trying to achieve with this sed command?

> **Comments:**
> **username_0**: It is attended that `sed 's/^/LP1-/' is putting LP1- at the front of every line because I would like to use [SGSautoSNP.py](https://www.mdpi.com/2079-7737/1/2/370/htm) to detect different cultivars. How is it possible to fix the above pipeline?  

---

