<p>This bash script will get you all of the reads that do not start with your primer sequence. It also handles degeneracy. It requires the tool <a href="https://github.com/lh3/bioawk" rel="nofollow noreferrer">bioawk</a>.</p>
<p>It works by looking at the first bases in the read and only prints out the headers that do not have a match with the degenerate primer. Then, <code>comm</code> makes sure that the read was retained in both the R1 and R2 filtering. This allows reads to be filtered whether or not the primers start at the beginning of the R1 or R2 read.</p>
<p><code>seqtk subseq</code> is where you recover the reads that don't start with the primer.</p>
<p>Be warned though, any off-by-one or sequencing errors will allow some of the sequences with primers at the beginning. Also, this script requires that the headers for the R1 and R2 reads are the same and aren't appended with anything like &quot;\1&quot;, &quot;\2&quot;, et cetera.</p>
<pre><code>#!/bin/bash
get_primer_free_starts () {
    bioawk -cfastx 'BEGIN{seqs[0] = &quot;CATGGGAGGGCGATCGCAA&quot;; seqs[1] = &quot;CCTGGGAGGGCGATCGCAA&quot;; \
        seqs[2] = &quot;CGTGGGAGGGCGATCGCAA&quot;; seqs[3] = &quot;CTTGGGAGGGCGATCGCAA&quot;} \
        {thisseq = substr(<span class="math-container">$seq, 1, length(seqs[0])); printme=1\
        for (i=0; i&lt;=3; i++) { \
            if (thisseq != seqs[i]){ \
                printme=0 \
            }
        }; if (printme==1){ \
            print($seq)
        }}' "$</span>{INPUT} | sort &gt; &quot;${OUTPUT}
}

INPUT=&quot;R1.fastq.gz&quot;
OUTPUT=&quot;R1_OK.txt&quot;
get_primer_free_starts

INPUT=&quot;R2.fastq.gz&quot;
OUTPUT=&quot;R2_OK.txt&quot;
get_primer_free_starts

comm -12 R1_OK.txt R2_OK.txt | sort | uniq &gt; passing_reads.txt

seqtk subseq R1.fastq.gz passing_reads.txt &gt; R1.filtered.fq
seqtk subseq R2.fastq.gz passing_reads.txt &gt; R2.filtered.fq
</code></pre>

CC BY-SA 4.0
2018-10-10T22:38:16.200Z
2022-06-09T13:50:31.020Z
2022-06-09T13:50:31.020Z
conchoecia
<h3>BACKGROUND</h3>
<p>I work on NGS data (illumina paired ends reads) coming from a full extract of RNA (metagenomic). We are interested in the viral fraction of this extract.</p>
<p>I observed a contamination with a PCR amplicon. This contamination has been identified as coming from a PCR result that we used to make near the library preparation. So I know exactly the primers used in the PCR.</p>
<p>Here you can see a tablet visualization illustrating the contamination (2 contaminations). I obtained that image with a mapping on a known reference of my virus. <a href="https://i.sstatic.net/92NDo.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/92NDo.png" alt="twin towers and world trade center" /></a></p>
<h3>PROBLEM</h3>
<p>The problem I have is that these PCR products are exactly the type of sequence I want from my extract (identifying part of the virus). So I have some &quot;false positive&quot; and some misidentification of viruses present in the extract.</p>
<p>I have thought about removing all reads containing primer sequences but it leads to a gap in my data and I can't identify the viruses in my extract anymore.
I am thinking that all of the read pairs that begin before or end after my primers are &quot;true-positives&quot; and I would really like to keep them (even having the primer sequence in them).</p>
<p><a href="https://i.sstatic.net/eIBzT.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/eIBzT.png" alt="enter image description here" /></a></p>
<p><strong>Do you know any tool that can help me in that situation?</strong> i.e. removing reads containing a sequence but only if it begins with it.</p>
<p><em>If the tool can also handle the degeneration of my primer it would be perfect.</em><br />
An example of primer I used is</p>
<pre><code>CNTGGGAGGGCGATCGCAA
</code></pre>
<p><strong>Complementary information (asked in comments)</strong><br />
I have around 30 million read pairs (60 millions reads in total). Contamination reads are between 200.000 to a 1 millions reads (depending on sample). &quot;true&quot; reads (not from the contamination) represent around 0.5% of the contamination.</p>

CC BY-SA 4.0
2018-10-10T15:34:10.140Z
Welcome to the site Untitpoi. Could you please clarify how many read do you remove when you "remove all reads containing primer sequences"? Also how many reads you have in total? Could you quantify the contamination (i.e 50% of your reads are contaminated)? Also how did you try to remove these sequence did you use a bash script or something like that?
llrs
CC BY-SA 4.0
2018-10-10T16:58:27.313Z
Have you searched this site yet?  There are other similar questions such as https://bioinformatics.stackexchange.com/questions/5163/extracting-sequences-from-fasta-beginning-with-common-5-end.   Also what tools have you tried?
Bioathlete
CC BY-SA 4.0
2018-10-10T17:15:48.803Z
Can you post an example of what you reasonably know to be a PCR contaminant and a reasonably similar read that you believe not to be but gets removed using the methods you've tried? That would help us fully grasp exactly what you're dealing with.
Devon Ryan
CC BY-SA 4.0
2018-10-11T07:49:36.057Z
@DevonRyan I added some graph to explain my problem, I hope this is clearer now. I didn't try anything yet but I will try a cut-adapt method today and the bioawk method proposed by conchoecia.
Untitpoi
CC BY-SA 4.0
2018-10-12T21:18:20.690Z
I just saw your post with the exact sequence of the forward and reverse primers. Is it true that you expect the beginning of one read to start with the forward primer, and the beginning of its paired read to start with the reverse primer? If so, I should change my answer to reflect that and to better filter your data.
conchoecia
<rna-seq><ngs><covid-19><pcr>
Tool to remove a PCR contamination in NGS data
# Tool to remove a PCR contamination in NGS data

(Asked by: username_0 on 2018-10-10)

BACKGROUND
I work on NGS data (illumina paired ends reads) coming from a full extract of RNA (metagenomic). We are interested in the viral fraction of this extract.
I observed a contamination with a PCR amplicon. This contamination has been identified as coming from a PCR result that we used to make near the library preparation. So I know exactly the primers used in the PCR.
Here you can see a tablet visualization illustrating the contamination (2 contaminations). I obtained that image with a mapping on a known reference of my virus. 
PROBLEM
The problem I have is that these PCR products are exactly the type of sequence I want from my extract (identifying part of the virus). So I have some "false positive" and some misidentification of viruses present in the extract.
I have thought about removing all reads containing primer sequences but it leads to a gap in my data and I can't identify the viruses in my extract anymore.
I am thinking that all of the read pairs that begin before or end after my primers are "true-positives" and I would really like to keep them (even having the primer sequence in them).

Do you know any tool that can help me in that situation? i.e. removing reads containing a sequence but only if it begins with it.
If the tool can also handle the degeneration of my primer it would be perfect.
An example of primer I used is
CNTGGGAGGGCGATCGCAA

Complementary information (asked in comments)
I have around 30 million read pairs (60 millions reads in total). Contamination reads are between 200.000 to a 1 millions reads (depending on sample). "true" reads (not from the contamination) represent around 0.5% of the contamination.

**Tags:** rna-seq, ngs, covid-19, pcr
> **Comments:**
> **username_1**: Welcome to the site Untitpoi. Could you please clarify how many read do you remove when you "remove all reads containing primer sequences"? Also how many reads you have in total? Could you quantify the contamination (i.e 50% of your reads are contaminated)? Also how did you try to remove these sequence did you use a bash script or something like that?  
> **username_2**: Have you searched this site yet?  There are other similar questions such as https://bioinformatics.stackexchange.com/questions/5163/extracting-sequences-from-fasta-beginning-with-common-5-end.   Also what tools have you tried?  
> **username_3**: Can you post an example of what you reasonably know to be a PCR contaminant and a reasonably similar read that you believe not to be but gets removed using the methods you've tried? That would help us fully grasp exactly what you're dealing with.  
> **username_0**: @DevonRyan I added some graph to explain my problem, I hope this is clearer now. I didn't try anything yet but I will try a cut-adapt method today and the bioawk method proposed by conchoecia.  
> **username_4**: I just saw your post with the exact sequence of the forward and reverse primers. Is it true that you expect the beginning of one read to start with the forward primer, and the beginning of its paired read to start with the reverse primer? If so, I should change my answer to reflect that and to better filter your data.  

---

## 1 answers

### Answer by username_4 on 2018-10-10. Score: 2

This bash script will get you all of the reads that do not start with your primer sequence. It also handles degeneracy. It requires the tool bioawk.
It works by looking at the first bases in the read and only prints out the headers that do not have a match with the degenerate primer. Then, comm makes sure that the read was retained in both the R1 and R2 filtering. This allows reads to be filtered whether or not the primers start at the beginning of the R1 or R2 read.
seqtk subseq is where you recover the reads that don't start with the primer.
Be warned though, any off-by-one or sequencing errors will allow some of the sequences with primers at the beginning. Also, this script requires that the headers for the R1 and R2 reads are the same and aren't appended with anything like "\1", "\2", et cetera.
#!/bin/bash
get_primer_free_starts () {
    bioawk -cfastx 'BEGIN{seqs[0] = "CATGGGAGGGCGATCGCAA"; seqs[1] = "CCTGGGAGGGCGATCGCAA"; \
        seqs[2] = "CGTGGGAGGGCGATCGCAA"; seqs[3] = "CTTGGGAGGGCGATCGCAA"} \
        {thisseq = substr($seq, 1, length(seqs[0])); printme=1\
        for (i=0; i<=3; i++) { \
            if (thisseq != seqs[i]){ \
                printme=0 \
            }
        }; if (printme==1){ \
            print($seq)
        }}' "${INPUT} | sort > "${OUTPUT}
}

INPUT="R1.fastq.gz"
OUTPUT="R1_OK.txt"
get_primer_free_starts

INPUT="R2.fastq.gz"
OUTPUT="R2_OK.txt"
get_primer_free_starts

comm -12 R1_OK.txt R2_OK.txt | sort | uniq > passing_reads.txt

seqtk subseq R1.fastq.gz passing_reads.txt > R1.filtered.fq
seqtk subseq R2.fastq.gz passing_reads.txt > R2.filtered.fq


---

