<p>The search term you need is <strong>[gene name]</strong> rather than <em>[title]</em>. For example,</p>
<pre><code>(Gasterosteus aculeatus[Organism] AND cox1[Gene Name]) OR (Gasterosteus aculeatus[Organism] AND coi[Gene Name]) 
</code></pre>
<p>Results in <strong>266</strong> hits which all contain COX1 COI</p>
<p>Using Daphnia magna ...</p>
<pre><code>(Daphnia magna[Organism] AND cox1[Gene Name]) OR (Daphnia magna[Organism] AND coi[Gene Name]) 
</code></pre>
<p>This results in <strong>775</strong> hits.</p>
<p><em>Daphina magna</em> comprises <strong>61</strong> mtDNA genomes, you perform the above search and sort by sequence length and set the sequences per page to 200(easy).</p>
<p>Hope that resolves the issue.</p>

CC BY-SA 4.0
2023-11-11T17:11:36.337Z
2023-11-11T17:19:14.627Z
2023-11-11T17:19:14.627Z
M__
<p>There are two main problems I can see in your approach. First, gene names are not standardized across species and the &quot;same&quot; gene (what that means is a whole different discussion) can have different names. For example, human  &quot;Cytochrome c oxidase assembly factor 1&quot; <a href="https://www.genenames.org/data/gene-symbol-report/#!/hgnc_id/HGNC:21868" rel="nofollow noreferrer">has the symbol &quot;COA1&quot;</a>.</p>
<p>Next, I don't quite understand why you expect the word &quot;complete&quot; to be in the title. That is actually why you don't find the <em>D. magna</em> sequence: it isn't complete, the start codon hasn't been correctly annotated as you can see on the <a href="https://www.ncbi.nlm.nih.gov/gene/24147205/" rel="nofollow noreferrer">relevant NCBI gene page</a>:</p>
<p><a href="https://i.sstatic.net/7xM2k.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/7xM2k.png" alt="Screenshot showing that the start codon is not determined" /></a></p>
<p>I would instead try something like this:</p>
<pre><code>((chrM[Chromosome]) OR (chromosome M[Chromosome]) OR (MT[Chromosome]) OR (chromosome MT[Chromosome])) AND ((CO1[Gene Name]) OR (COX1[Gene Name]) OR ((cytochrome[Gene Name]) AND (oxidase[Gene Name]))
</code></pre>
<p>or</p>
<pre><code>((16S[Gene/Protein Name]) and (ribosomal[Gene/Protein Name]))
</code></pre>
<p>All that said, trying to rely on gene annotations is rarely the best approach. There just isn't enough standardization. I would instead look at databases that provide lists of homologous genes. For instance, NCBI's <a href="https://www.ncbi.nlm.nih.gov/research/cog/" rel="nofollow noreferrer">Clusters of Orthologous Genes (COGs)</a>, which provides grouped homologous sequences.</p>

CC BY-SA 4.0
2023-11-17T14:21:55.817Z
Thanks for the tips they are very relevant. But the COG is only a viable option for archea and bacteria so it doesn't suit for me.
Mirko
CC BY-SA 4.0
2023-11-17T14:38:40.373Z
Ah, crud, sorry @Mirko, I didn't know it was limited to microorganisms. There is also [KOG](https://www.hsls.pitt.edu/obrc/index.php?page=URL1144075392) and [Ortho MCL](https://orthomcl.org/orthomcl/app) or you can use BioMart to extract orthologs etc. My main point here is that trying to use a gene name is unlikely to work well and you should look for dedicated resources instead.
terdon
CC BY-SA 4.0
2023-11-11T13:53:43.673Z
2023-11-13T16:02:07.980Z
2023-11-13T16:02:07.980Z
terdon
<p>I want to retrieve a sequence for many species from the Nucleotide database in NCBI.</p>
<p>I'm using a command line approach and I have to figure out what is the best query that will return exclusively the region of DNA I am interested in and filter out unwanted noisy sequences.</p>
<p>I am using a wide range of species (that I have stored as a list of TaxIDs of about 2000 species): it includes small crustaceans, invertebrates, some algae and small vertebrates (reptiles, amphibians and few mammals).</p>
<p>My final goal is to obtain a phylogenetic tree for all or most of the species.</p>
<p>I have been suggested to use these genes to create such phylogeny</p>
<ul>
<li>CO1 mithocondrial DNA</li>
<li>16S rRNA</li>
<li>18S rRNA</li>
</ul>
<p>I want to formulate a specific query that will return exclusively those sequences.</p>
<p>I'm using the GenBank query builder to visually check for the accuracy of my search and when I find a good query I will use it in the API.</p>
<p>So far I came up with the following queries:</p>
<ul>
<li><code>(COX1[Title] OR CO1[Title]) AND complete[Title]</code> --&gt; 63/2000 Species</li>
<li><code>16S[title] AND complete[title] AND rRNA[title] NOT partial[title]</code> --&gt; 9/2000 species</li>
<li><code>18S[title] AND complete[title] AND rRNA[title] NOT partial[title]</code> --&gt; 15/2000 species</li>
</ul>
<p>As you can see the number of species that I get sequence for is very low compared to the initial 2000 species. I doubt that we have so little available sequences (especially for COX1 that is used for barcoding)</p>
<p>Can you help me understanding whether my queries are good or not? And if possible suggest a better alternative</p>
<h3>More Info</h3>
<p>A subset of 10 of my species of interest is</p>
<pre><code>Rasbora heteromorpha
Elasmopus rapax
Gasterosteus aculeatus
Palaemonetes pugio
Catostomus commersoni
Daphnia magna
Oryzias latipes
Xenopus laevis
Tigriopus japonicus
Oncorhynchus mykiss
</code></pre>
<p>Out of these 10 only 2 have sequences available using the COX1 query</p>
<div class="s-table-container">
<table class="s-table">
<thead>
<tr>
<th>Name</th>
<th>SeqID</th>
</tr>
</thead>
<tbody>
<tr>
<td>Oncorhynchus mykiss</td>
<td>EU186789.1</td>
</tr>
<tr>
<td>Xenopus laevis</td>
<td>AB278691.1</td>
</tr>
</tbody>
</table>
</div>
<p>But Daphnia magna is one of the most commonly used organisms in lab and I found <a href="https://www.tandfonline.com/doi/full/10.1080/23802359.2020.1750995" rel="nofollow noreferrer">this paper</a> regarding the complete mitogenome of a specific strain of D. magna thus implying that the complete mitogenome of the species is already known. That means that there must be a way to retrieve all the other species that have full mitogenome but no COX1 gene mentioned in their GenBank title.</p>

CC BY-SA 4.0
2023-11-09T09:52:45.030Z
What database are you searching? I get 1995 hits for `16S[title] AND complete[title] AND rRNA[title] NOT partial[title]`, across 1430 eukaryotes, 287 bacteria and 178 archaea when using "nucleotide".
terdon
CC BY-SA 4.0
2023-11-09T10:53:21.193Z
Yes sorry maybe i was not very clear: the numbers on the right are the species that are both in my database and also in the results from Nucleotide. So even if you get 1400 eukariotes only 9 of these are overlapping with the species i work with. But they are mostly lab species and not rare, That is why I find it strange that i don't get many sequences from my search
Mirko
CC BY-SA 4.0
2023-11-11T13:07:27.367Z
You seem to be assuming gene names are shared across organisms. It's a reasonable assumption but, sadly, it is absolutely false. Gene names are _not_ shared across species. They can be, sometimes, but not always or even often. For example, there is no `CO1` or `COX1` in human. "Cytochrome c oxidase assembly factor 1" [has the symbol "COA1"](https://www.genenames.org/data/gene-symbol-report/#!/hgnc_id/HGNC:21868).
terdon
CC BY-SA 4.0
2023-11-17T14:23:05.810Z
For anyone having the same type of problem as me (batch COI download) i recommend reading this article as it has proven very useful for me DOI: 10.1111/1755-0998.13756
Mirko
<phylogenetics><phylogeny><sequence-annotation><genbank><barcode>
What is the best Query to retrieve DNA from NCBI?
# What is the best Query to retrieve DNA from NCBI?

(Asked by: username_0 on 2023-11-08)

I want to retrieve a sequence for many species from the Nucleotide database in NCBI.
I'm using a command line approach and I have to figure out what is the best query that will return exclusively the region of DNA I am interested in and filter out unwanted noisy sequences.
I am using a wide range of species (that I have stored as a list of TaxIDs of about 2000 species): it includes small crustaceans, invertebrates, some algae and small vertebrates (reptiles, amphibians and few mammals).
My final goal is to obtain a phylogenetic tree for all or most of the species.
I have been suggested to use these genes to create such phylogeny

CO1 mithocondrial DNA
16S rRNA
18S rRNA

I want to formulate a specific query that will return exclusively those sequences.
I'm using the GenBank query builder to visually check for the accuracy of my search and when I find a good query I will use it in the API.
So far I came up with the following queries:

(COX1[Title] OR CO1[Title]) AND complete[Title] --> 63/2000 Species
16S[title] AND complete[title] AND rRNA[title] NOT partial[title] --> 9/2000 species
18S[title] AND complete[title] AND rRNA[title] NOT partial[title] --> 15/2000 species

As you can see the number of species that I get sequence for is very low compared to the initial 2000 species. I doubt that we have so little available sequences (especially for COX1 that is used for barcoding)
Can you help me understanding whether my queries are good or not? And if possible suggest a better alternative
More Info
A subset of 10 of my species of interest is
Rasbora heteromorpha
Elasmopus rapax
Gasterosteus aculeatus
Palaemonetes pugio
Catostomus commersoni
Daphnia magna
Oryzias latipes
Xenopus laevis
Tigriopus japonicus
Oncorhynchus mykiss

Out of these 10 only 2 have sequences available using the COX1 query




Name
SeqID




Oncorhynchus mykiss
EU186789.1


Xenopus laevis
AB278691.1




But Daphnia magna is one of the most commonly used organisms in lab and I found this paper regarding the complete mitogenome of a specific strain of D. magna thus implying that the complete mitogenome of the species is already known. That means that there must be a way to retrieve all the other species that have full mitogenome but no COX1 gene mentioned in their GenBank title.

**Tags:** phylogenetics, phylogeny, sequence-annotation, genbank, barcode
> **Comments:**
> **username_1**: What database are you searching? I get 1995 hits for `16S[title] AND complete[title] AND rRNA[title] NOT partial[title]`, across 1430 eukaryotes, 287 bacteria and 178 archaea when using "nucleotide".  
> **username_0**: Yes sorry maybe i was not very clear: the numbers on the right are the species that are both in my database and also in the results from Nucleotide. So even if you get 1400 eukariotes only 9 of these are overlapping with the species i work with. But they are mostly lab species and not rare, That is why I find it strange that i don't get many sequences from my search  
> **username_1**: You seem to be assuming gene names are shared across organisms. It's a reasonable assumption but, sadly, it is absolutely false. Gene names are _not_ shared across species. They can be, sometimes, but not always or even often. For example, there is no `CO1` or `COX1` in human. "Cytochrome c oxidase assembly factor 1" [has the symbol "COA1"](https://www.genenames.org/data/gene-symbol-report/#!/hgnc_id/HGNC:21868).  
> **username_0**: For anyone having the same type of problem as me (batch COI download) i recommend reading this article as it has proven very useful for me DOI: 10.1111/1755-0998.13756  

---

## 2 answers

### Answer by username_2 on 2023-11-11. Score: 1 | Accepted answer

The search term you need is [gene name] rather than [title]. For example,
(Gasterosteus aculeatus[Organism] AND cox1[Gene Name]) OR (Gasterosteus aculeatus[Organism] AND coi[Gene Name]) 

Results in 266 hits which all contain COX1 COI
Using Daphnia magna ...
(Daphnia magna[Organism] AND cox1[Gene Name]) OR (Daphnia magna[Organism] AND coi[Gene Name]) 

This results in 775 hits.
Daphina magna comprises 61 mtDNA genomes, you perform the above search and sort by sequence length and set the sequences per page to 200(easy).
Hope that resolves the issue.


---

### Answer by username_1 on 2023-11-11. Score: 2

There are two main problems I can see in your approach. First, gene names are not standardized across species and the "same" gene (what that means is a whole different discussion) can have different names. For example, human  "Cytochrome c oxidase assembly factor 1" has the symbol "COA1".
Next, I don't quite understand why you expect the word "complete" to be in the title. That is actually why you don't find the D. magna sequence: it isn't complete, the start codon hasn't been correctly annotated as you can see on the relevant NCBI gene page:

I would instead try something like this:
((chrM[Chromosome]) OR (chromosome M[Chromosome]) OR (MT[Chromosome]) OR (chromosome MT[Chromosome])) AND ((CO1[Gene Name]) OR (COX1[Gene Name]) OR ((cytochrome[Gene Name]) AND (oxidase[Gene Name]))

or
((16S[Gene/Protein Name]) and (ribosomal[Gene/Protein Name]))

All that said, trying to rely on gene annotations is rarely the best approach. There just isn't enough standardization. I would instead look at databases that provide lists of homologous genes. For instance, NCBI's Clusters of Orthologous Genes (COGs), which provides grouped homologous sequences.

> **Comments:**
> **username_0**: Thanks for the tips they are very relevant. But the COG is only a viable option for archea and bacteria so it doesn't suit for me.  
> **username_1**: Ah, crud, sorry @Mirko, I didn't know it was limited to microorganisms. There is also [KOG](https://www.hsls.pitt.edu/obrc/index.php?page=URL1144075392) and [Ortho MCL](https://orthomcl.org/orthomcl/app) or you can use BioMart to extract orthologs etc. My main point here is that trying to use a gene name is unlikely to work well and you should look for dedicated resources instead.  

---

