<p>After modifying the code above in the original post, this one works the best for me. I got it working after putting 'not' in the second last line instead of &quot;if record.id in to keep&quot; to &quot;if record.id not in to keep&quot;:</p>
<pre><code>from Bio import SeqIO
with open('headers.txt', 'r') as fin:
    to_remove = set([])
    for line in fin:
        to_remove.add(line.strip().split(' ')[0])
with open('output.fa', 'w') as fout:
    for record in SeqIO.parse('sample.fa','fasta'):
        if record.id in to_remove:
            pass
        else:
            SeqIO.write(record, fout, 'fasta')
</code></pre>

CC BY-SA 4.0
2023-12-21T05:41:14.237Z
2023-12-21T09:15:31.863Z
2023-12-21T09:15:31.863Z
Irfan
<p>You can use something like seqkit: <a href="https://bioinf.shenwei.me/seqkit/" rel="nofollow noreferrer">https://bioinf.shenwei.me/seqkit/</a></p>
<p>seqkit has a <code>grep</code> sub-command that is one of the most flexible I've ever seen. Your use case is quite simple, you can use <code>seqkit grep -v</code> like so:</p>
<pre><code>seqkit grep -v -n -f list_of_IDs_to_exclude.txt in.fasta &gt; out.fasta
</code></pre>

CC BY-SA 4.0
2023-12-20T19:23:48.367Z
2023-12-21T09:16:21.220Z
2023-12-21T09:16:21.220Z
Ram RS
<p>This should work</p>
<pre><code>import argparse

ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.ArgumentDefaultsHelpFormatter)
ap.add_argument('--fasta', required=True, type=str, help='Fasta file to extract from')
ap.add_argument('--headers', required=True, type=str, help='Headers to remove, one per line.')

conf = ap.parse_args()
f = conf.fasta
h = conf.headers
header_set = set()

with open(h) as header_file:
    for line in header_file:
        line = line.rstrip()
        header_set.add(line)

with open(f) as fasta_file:
    remove_sequence = False
    for line in fasta_file:
        if '&gt;' in line:
            header_id = line.split()[0].replace(&quot;&gt;&quot;, &quot;&quot;)
            if header_id in header_set:
                remove_sequence = True
            else:
                remove_sequence = False
                print(line.rstrip())
        elif not remove_sequence:
            print(line.rstrip())

exit()
</code></pre>
<p>Save it as python script and then run</p>
<p><code>python remove_fasta.py --fasta sample.fa --headers listofheaders.txt &gt; cleaned.fasta</code></p>

CC BY-SA 4.0
2023-12-21T02:23:29.083Z
Thanks. I tried this first thing in the morning. First it ran but didn't remove anything, then I removed '>' character from the headers list and it gave the correct expected cleaned file.
What if I want to run it directly form the python IDLE and not from cmd?
Irfan
CC BY-SA 4.0
2023-12-21T10:29:09.060Z
I'm glad it worked. As you can see, there are other options that will get the job done, but I understood that you want to learn how to do this in Python. I don't personally use the Python IDLE on Windows, but I believe you can save your code, open it in IDLE, and click on "run module". Make sure to modify the file locations as needed. I hope this helps, and good luck with your coding
CorteZero
CC BY-SA 4.0
2023-12-20T10:05:33.050Z
2023-12-20T10:12:09.957Z
2023-12-20T10:12:09.957Z
CorteZero
<p>I would argue writing a whole python script for this is overkill. Assuming you are using a*nix based operating system like Linux or macOS, you can use the two scripts <a href="https://bioinformatics.stackexchange.com/a/2651/298">I have posted here</a>, <code>FastaToTbl</code> and <code>TblToFasta</code>, and just <code>grep</code> out the ones you do not want:</p>
<pre><code>FastaToTbl sample.fa | grep -v -xFf listofheaders.txt | TblToFasta &gt; cleaned.fa
</code></pre>

CC BY-SA 4.0
2023-12-21T01:57:25.487Z
Hi terdon, I am only using windows 10 PC with python. I am trying the above suggestions as well. Writing my own code is a big opportunity to learn as well.
Irfan
CC BY-SA 4.0
2023-12-21T09:43:36.170Z
Oh. Windows? OK. But then please add that to your question. Windows isn't really used by bioinformaticians at all so nobody would guess you are trying this in windows. If you want to learn bioinformatics, I would urge you to use Linux or macOS or basically anything other than Windows.
terdon
CC BY-SA 4.0
2023-12-22T00:59:53.343Z
I agree with you. I am already on to it. ")
Irfan
CC BY-SA 4.0
2023-12-20T13:53:21.690Z
2023-12-20T13:53:21.690Z
terdon
<p>a python beginner here.
I have a fasta file with 2500+ sequences, and after doing some analysis I want to remove around 200+ sequences based on the matching IDs. Now, I have one fasta file (as sample.fa) and a text file with a list of IDs for the sequences that I want to remove from this fasta file. Below is an example of the fasta sequences. For simplicity, I have shortened the sequences and their respective IDs. The ID in the text file (for the sequences to remove, and in the fast file will be exactly same). The sequences were originally downloaded form NCBI and had very long headers so I have them shortened as at this point I don't need all that info, usually just the sequence ID in the headers is enough.</p>
<p>Sample fasta sequences in fasta file:</p>
<pre><code>&gt;WP_017417394.1-Bacillus_amyloliquefaciens_gp.
MRGKKVWISLLFALALIFTMAFGSTSPAQAAGKSNGEKKYIVGFKQTMSTMSAAKKKDVISEKGGKVQKQFKYVDAASAT
&gt;WP_268291751.1-Bacillus_inaquosorum
MRRKKLWISLLFALTLIFTMAFSNMSAQAAGKSSTEKKYIVGFKETMSAMSSAKKKDVISEKGGKVQKQFKYVNAATATL
&gt;WP_003327717.1-Bacillus_atrophaeus
MRSRKLWIGLLFAFTLVFTMAFSSMSPAQAAGKSSEEKKYIVGFKQTMGAMSAAAKKDVIAENGGKLQKQFKYVDAATAT
&gt;WP_206702175.1-Bacillus_vallismortis
MAFSNMSAQAAGKSSTEKKYIVGFKQTMSVMSSAKKKDVISEKGGKVQKQFKYVNAAAATLDAKAVKELKQDPSVAYVEE
&gt;KAF1680298.1-Bacillus_sp.
MKSKKLWISLLFALTLVFTMAFSNMSAQASGKNSTEKKYIVGFKQTMSTAKKKDIISEKGGKVQKQFKYVNAASATLDQK
&gt;WP_172293960.1-Bacillus_sp.
MGKKRLWYSLLTVLLLVFSMAFSHPANAVKSSAKDVKKDYIVGFKSSLKTTSAKKDVIKENGGKVDKHFRSINAAKVTLD
&gt;WP_082194748.1-Bacillus_xiamenensis
MCVKKKNVMTSVLLAVPLLFSAGFGGSMANAETASKSESEKSYIVGFKASATTNSSKKQAVTQNGGTLEKQYRLINAAQV
&gt;WP_048353666.1-Bacillus_glycinifermentans
MKKRSLWLSVLTALLLVFTMAFSNPASAEQPAKDVEKDYIVGFKSSVKTASVKKDVIKESGGKVDKQFKIINAAKATLDQ
&gt;WP_206702858.1-Bacillus_sonorensis
MTALLLVLSTAFSSPASAAQPAKDVEKDYIVGFKSSVKTAAVKKDVIKENGGKVDKQFKIINAAKATLDQEEVKALKKD
</code></pre>
<p>The list of IDs in a separate text file that need to be removed from the above fasta file:</p>
<pre><code>&gt;WP_017417394.1-Bacillus_amyloliquefaciens_gp.
&gt;WP_003327717.1-Bacillus_atrophaeus
&gt;KAF1680298.1-Bacillus_sp.
&gt;WP_048353666.1-Bacillus_glycinifermentans
&gt;WP_206702858.1-Bacillus_sonorensis
</code></pre>
<p>I searched and tried a few scripts shared elsewhere (e.g. from <a href="https://bioinformatics.stackexchange.com/questions/15326/how-to-remove-sequences-from-a-fasta-file-using-a-sequence-id-list-which-contain">here</a>), but I couldn't make it work with my files. So all the help will be really appreciated. Thanks in advance.</p>
<p>PS. I am using using python usually with its own IDLE on windows 10 PC.</p>
<p>I did one thing with <a href="https://bioinformatics.stackexchange.com/questions/15393/filtering-sequences-entries-by-headers-id-from-a-fasta-file-database">this code</a>. I removed the '&gt;' character from the headers file and then this code works but it keeps those headers and their respective sequences. I think it works as intended in this code, but I want to remove those from the sequence file not to keep.</p>
<p>Update: as asked below, I copied this code from another post.</p>
<pre><code>from Bio import SeqIO
with open('headers.txt', 'r') as fin:
    to_keep = set([])
    for line in fin:
        to_keep.add(line.strip().split(' ')[0])
with open('output.fa', 'w') as fout:
    for record in SeqIO.parse('sample.fa','fasta'):
        if record.id in to_keep:
            SeqIO.write(record, fout, 'fasta')
</code></pre>

CC BY-SA 4.0
2023-12-20T17:29:39.290Z
And how did they not work? I was about to write an answer but it is essentially what  had already written as an answer to the other question. Please [edit] and clarify why those other solutions didn't work, since right now, this looks like a duplicate question.
terdon
CC BY-SA 4.0
2023-12-21T16:49:10.823Z
Does this answer your question? [Filtering Sequences (entries) by headers ID from a Fasta file database](https://bioinformatics.stackexchange.com/questions/15393/filtering-sequences-entries-by-headers-id-from-a-fasta-file-database)
Ram RS
CC BY-SA 4.0
2023-12-25T20:15:41.473Z
This is not a good FASTA file format. IDs and description should be separate by space. I suggest to take a look at the official documentation. Good practice will make your life easier here an [example](https://www.ncbi.nlm.nih.gov/genbank/fastaformat/) [or](https://bioperl.org/formats/sequence_formats/FASTA_sequence_format)
Supertech
CC BY-SA 4.0
2023-12-28T05:55:18.377Z
yes, I agree. but I do that on purpose b/c after some manipulation, I make a phylogenetic tree and visualize (usually on ITOL) which messes up the formatting, sometimes it is very long and I am interested to see only the accession ID and the bacterial name. So I delete the rest and make it as one continuous string so everything I need stays in and still within a short limit to be visible.
Irfan
<python><fasta><sequence>
Remove sequences from a fasta file with IDs from a text file using Python
# Remove sequences from a fasta file with IDs from a text file using Python

(Asked by: username_0 on 2023-12-20)

a python beginner here.
I have a fasta file with 2500+ sequences, and after doing some analysis I want to remove around 200+ sequences based on the matching IDs. Now, I have one fasta file (as sample.fa) and a text file with a list of IDs for the sequences that I want to remove from this fasta file. Below is an example of the fasta sequences. For simplicity, I have shortened the sequences and their respective IDs. The ID in the text file (for the sequences to remove, and in the fast file will be exactly same). The sequences were originally downloaded form NCBI and had very long headers so I have them shortened as at this point I don't need all that info, usually just the sequence ID in the headers is enough.
Sample fasta sequences in fasta file:
>WP_017417394.1-Bacillus_amyloliquefaciens_gp.
MRGKKVWISLLFALALIFTMAFGSTSPAQAAGKSNGEKKYIVGFKQTMSTMSAAKKKDVISEKGGKVQKQFKYVDAASAT
>WP_268291751.1-Bacillus_inaquosorum
MRRKKLWISLLFALTLIFTMAFSNMSAQAAGKSSTEKKYIVGFKETMSAMSSAKKKDVISEKGGKVQKQFKYVNAATATL
>WP_003327717.1-Bacillus_atrophaeus
MRSRKLWIGLLFAFTLVFTMAFSSMSPAQAAGKSSEEKKYIVGFKQTMGAMSAAAKKDVIAENGGKLQKQFKYVDAATAT
>WP_206702175.1-Bacillus_vallismortis
MAFSNMSAQAAGKSSTEKKYIVGFKQTMSVMSSAKKKDVISEKGGKVQKQFKYVNAAAATLDAKAVKELKQDPSVAYVEE
>KAF1680298.1-Bacillus_sp.
MKSKKLWISLLFALTLVFTMAFSNMSAQASGKNSTEKKYIVGFKQTMSTAKKKDIISEKGGKVQKQFKYVNAASATLDQK
>WP_172293960.1-Bacillus_sp.
MGKKRLWYSLLTVLLLVFSMAFSHPANAVKSSAKDVKKDYIVGFKSSLKTTSAKKDVIKENGGKVDKHFRSINAAKVTLD
>WP_082194748.1-Bacillus_xiamenensis
MCVKKKNVMTSVLLAVPLLFSAGFGGSMANAETASKSESEKSYIVGFKASATTNSSKKQAVTQNGGTLEKQYRLINAAQV
>WP_048353666.1-Bacillus_glycinifermentans
MKKRSLWLSVLTALLLVFTMAFSNPASAEQPAKDVEKDYIVGFKSSVKTASVKKDVIKESGGKVDKQFKIINAAKATLDQ
>WP_206702858.1-Bacillus_sonorensis
MTALLLVLSTAFSSPASAAQPAKDVEKDYIVGFKSSVKTAAVKKDVIKENGGKVDKQFKIINAAKATLDQEEVKALKKD

The list of IDs in a separate text file that need to be removed from the above fasta file:
>WP_017417394.1-Bacillus_amyloliquefaciens_gp.
>WP_003327717.1-Bacillus_atrophaeus
>KAF1680298.1-Bacillus_sp.
>WP_048353666.1-Bacillus_glycinifermentans
>WP_206702858.1-Bacillus_sonorensis

I searched and tried a few scripts shared elsewhere (e.g. from here), but I couldn't make it work with my files. So all the help will be really appreciated. Thanks in advance.
PS. I am using using python usually with its own IDLE on windows 10 PC.
I did one thing with this code. I removed the '>' character from the headers file and then this code works but it keeps those headers and their respective sequences. I think it works as intended in this code, but I want to remove those from the sequence file not to keep.
Update: as asked below, I copied this code from another post.
from Bio import SeqIO
with open('headers.txt', 'r') as fin:
    to_keep = set([])
    for line in fin:
        to_keep.add(line.strip().split(' ')[0])
with open('output.fa', 'w') as fout:
    for record in SeqIO.parse('sample.fa','fasta'):
        if record.id in to_keep:
            SeqIO.write(record, fout, 'fasta')

**Tags:** python, fasta, sequence
> **Comments:**
> **username_1**: And how did they not work? I was about to write an answer but it is essentially what  had already written as an answer to the other question. Please [edit] and clarify why those other solutions didn't work, since right now, this looks like a duplicate question.  
> **username_2**: Does this answer your question? [Filtering Sequences (entries) by headers ID from a Fasta file database](https://bioinformatics.stackexchange.com/questions/15393/filtering-sequences-entries-by-headers-id-from-a-fasta-file-database)  
> **username_3**: This is not a good FASTA file format. IDs and description should be separate by space. I suggest to take a look at the official documentation. Good practice will make your life easier here an [example](https://www.ncbi.nlm.nih.gov/genbank/fastaformat/) [or](https://bioperl.org/formats/sequence_formats/FASTA_sequence_format)  
> **username_0**: yes, I agree. but I do that on purpose b/c after some manipulation, I make a phylogenetic tree and visualize (usually on ITOL) which messes up the formatting, sometimes it is very long and I am interested to see only the accession ID and the bacterial name. So I delete the rest and make it as one continuous string so everything I need stays in and still within a short limit to be visible.  

---

## 4 answers

### Answer by username_0 on 2023-12-21. Score: 2 | Accepted answer

After modifying the code above in the original post, this one works the best for me. I got it working after putting 'not' in the second last line instead of "if record.id in to keep" to "if record.id not in to keep":
from Bio import SeqIO
with open('headers.txt', 'r') as fin:
    to_remove = set([])
    for line in fin:
        to_remove.add(line.strip().split(' ')[0])
with open('output.fa', 'w') as fout:
    for record in SeqIO.parse('sample.fa','fasta'):
        if record.id in to_remove:
            pass
        else:
            SeqIO.write(record, fout, 'fasta')


---

### Answer by username_2 on 2023-12-20. Score: 5

You can use something like seqkit: https://bioinf.shenwei.me/seqkit/
seqkit has a grep sub-command that is one of the most flexible I've ever seen. Your use case is quite simple, you can use seqkit grep -v like so:
seqkit grep -v -n -f list_of_IDs_to_exclude.txt in.fasta > out.fasta


---

### Answer by username_4 on 2023-12-20. Score: 2

This should work
import argparse

ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.ArgumentDefaultsHelpFormatter)
ap.add_argument('--fasta', required=True, type=str, help='Fasta file to extract from')
ap.add_argument('--headers', required=True, type=str, help='Headers to remove, one per line.')

conf = ap.parse_args()
f = conf.fasta
h = conf.headers
header_set = set()

with open(h) as header_file:
    for line in header_file:
        line = line.rstrip()
        header_set.add(line)

with open(f) as fasta_file:
    remove_sequence = False
    for line in fasta_file:
        if '>' in line:
            header_id = line.split()[0].replace(">", "")
            if header_id in header_set:
                remove_sequence = True
            else:
                remove_sequence = False
                print(line.rstrip())
        elif not remove_sequence:
            print(line.rstrip())

exit()

Save it as python script and then run
python remove_fasta.py --fasta sample.fa --headers listofheaders.txt > cleaned.fasta

> **Comments:**
> **username_0**: Thanks. I tried this first thing in the morning. First it ran but didn't remove anything, then I removed '>' character from the headers list and it gave the correct expected cleaned file. What if I want to run it directly form the python IDLE and not from cmd?  
> **username_4**: I'm glad it worked. As you can see, there are other options that will get the job done, but I understood that you want to learn how to do this in Python. I don't personally use the Python IDLE on Windows, but I believe you can save your code, open it in IDLE, and click on "run module". Make sure to modify the file locations as needed. I hope this helps, and good luck with your coding  

---

### Answer by username_1 on 2023-12-20. Score: 2

I would argue writing a whole python script for this is overkill. Assuming you are using a*nix based operating system like Linux or macOS, you can use the two scripts I have posted here, FastaToTbl and TblToFasta, and just grep out the ones you do not want:
FastaToTbl sample.fa | grep -v -xFf listofheaders.txt | TblToFasta > cleaned.fa

> **Comments:**
> **username_0**: Hi terdon, I am only using windows 10 PC with python. I am trying the above suggestions as well. Writing my own code is a big opportunity to learn as well.  
> **username_1**: Oh. Windows? OK. But then please add that to your question. Windows isn't really used by bioinformaticians at all so nobody would guess you are trying this in windows. If you want to learn bioinformatics, I would urge you to use Linux or macOS or basically anything other than Windows.  
> **username_0**: I agree with you. I am already on to it. ")  

---

