<p>Your code (as currently formatted) should generate the correct output, but it is not efficient. Make <code>seen</code> a set to improve these O(n) <code>in</code> calls to O(1) time complexity- this will speed it up a lot</p>
<pre><code>seen = set()
records = []

for record in SeqIO.parse(&quot;b4r2.fasta&quot;, &quot;fasta&quot;):  
    if record.seq not in seen:
        seen.add(record.seq)
        records.append(record)
</code></pre>

CC BY-SA 4.0
2020-07-03T17:02:00.100Z
Thank you for your input!
User
CC BY-SA 4.0
2020-07-02T07:35:05.583Z
2020-07-02T07:35:05.583Z
Chris_Rands
<p>I'm attempting to remove redundant sequences from a fasta file (from NCBI). When I execute this code, it returns the number of spots, not the number of sequences. (Number of spots: 408,293, Number of sequences: 40,830)</p>
<p>Please let me know if you have run a code that can remove the duplicate ideas or have any ideas/suggestions.</p>
<p>Thank you SO much.</p>
<pre><code>from Bio import SeqIO
import time

start = time.time() 

seen = []
records = []

for record in SeqIO.parse(&quot;b4r2.fasta&quot;, &quot;fasta&quot;):  
    if str(record.seq) not in seen:
        seen.append(str(record.seq))
        records.append(record)


#writing to a fasta file
SeqIO.write(records, &quot;no_dupes_b4r2.fasta&quot;, &quot;fasta&quot;)
end = time.time()

print(f&quot;Run time is {(end- start)/60}&quot;) 
</code></pre>

CC BY-SA 4.0
2020-07-01T23:46:51.950Z
Hi, your code was not formatting and it became a bit of a headache to format. Please invest some more effort into presenting your questions better.
Ram RS
CC BY-SA 4.0
2020-07-02T18:41:55.210Z
Thank you so much, I'm just learning to use Stack Exchange! I appreciate it
User
<fasta><python><sequence>
Remove Redundant Sequences from FASTA file in Python
# Remove Redundant Sequences from FASTA file in Python

(Asked by: username_0 on 2020-07-01)

I'm attempting to remove redundant sequences from a fasta file (from NCBI). When I execute this code, it returns the number of spots, not the number of sequences. (Number of spots: 408,293, Number of sequences: 40,830)
Please let me know if you have run a code that can remove the duplicate ideas or have any ideas/suggestions.
Thank you SO much.
from Bio import SeqIO
import time

start = time.time() 

seen = []
records = []

for record in SeqIO.parse("b4r2.fasta", "fasta"):  
    if str(record.seq) not in seen:
        seen.append(str(record.seq))
        records.append(record)


#writing to a fasta file
SeqIO.write(records, "no_dupes_b4r2.fasta", "fasta")
end = time.time()

print(f"Run time is {(end- start)/60}")

**Tags:** fasta, python, sequence
> **Comments:**
> **username_1**: Hi, your code was not formatting and it became a bit of a headache to format. Please invest some more effort into presenting your questions better.  
> **username_0**: Thank you so much, I'm just learning to use Stack Exchange! I appreciate it  

---

## 1 answers

### Answer by username_2 on 2020-07-02. Score: 5 | Accepted answer

Your code (as currently formatted) should generate the correct output, but it is not efficient. Make seen a set to improve these O(n) in calls to O(1) time complexity- this will speed it up a lot
seen = set()
records = []

for record in SeqIO.parse("b4r2.fasta", "fasta"):  
    if record.seq not in seen:
        seen.add(record.seq)
        records.append(record)

> **Comments:**
> **username_0**: Thank you for your input!  

---

