<p>That file is actually trivial to parse. You only care about lines that have <code>N=$species</code>, so you can simply do:</p>

<pre><code>sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist 
</code></pre>

<p>That will return a tab separated list of taxID and species:</p>

<pre><code>$ sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist | head
648330  Aedes albopictus densovirus (isolate Boublik/1994)
10804   Adeno-associated virus 2
648242  Adeno-associated virus 2 (isolate Srivastava/1982)
118452  Abacion magnum
72259   Abaeis nicippe
102642  Abax parallelepipedus
392897  Abalistes stellaris
75332   Abbottina rivularis
515833  Abdopus aculeatus
56673   Antarctic bacterium DS2-3R
</code></pre>

<p>So to get human, for example, you could do:</p>

<pre><code>$ sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist | grep sapiens
63221   Homo sapiens neanderthalensis
9606    Homo sapiens
</code></pre>

<p>And, for exact matches:</p>

<pre><code>$ sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist | 
    awk -F'\t' '$2=="Homo sapiens"'
9606    Homo sapiens
</code></pre>

<p>If you also want to keep the taxon node (kingdom), you can use:</p>

<pre><code>sed -En 's/.* ([A-Z])  *([0-9]+): N=(.*)/\1\t\2\t\3/p' speclist 
</code></pre>

<p>Which prints:</p>

<pre><code>$ sed -En 's/.* ([A-Z])  *([0-9]+): N=(.*)/\1\t\2\t\3/p' speclist  | head
V   648330  Aedes albopictus densovirus (isolate Boublik/1994)
V   10804   Adeno-associated virus 2
V   648242  Adeno-associated virus 2 (isolate Srivastava/1982)
E   118452  Abacion magnum
E   72259   Abaeis nicippe
E   102642  Abax parallelepipedus
E   392897  Abalistes stellaris
E   75332   Abbottina rivularis
E   515833  Abdopus aculeatus
B   56673   Antarctic bacterium DS2-3R
</code></pre>

<hr>

<p>The <code>sed</code> command might appear a little daunting if you don't know <code>sed</code>, but it's actually quite simple. The <code>-E</code> flag enables extended regular expressions which give us a simpler syntax. The <code>-n</code> suppresses normal output, so that <code>sed</code> won't actually print anything unless explicitly told to. </p>

<p>The <code>s/from/to/</code> is the substitution operator and will replace <code>from</code> with <code>to</code>. Here, we are replacing zero or more characters (<code>.*</code>) until a space followed by one or more numbers (<code>[0-9]+</code>), a <code>:</code>, another space, then the string <code>N=</code> and then everything else (<code>.*</code>). The parentheses let us <em>capture</em> the matched strings, so that we can then refer to the 1st captured string as <code>\1</code>, the second as <code>\2</code> etc. We are therefore replacing the entire line (because the regex matches the entire line) with the 1st pattern captured (the taxID, <code>\1</code>), a tab (<code>\t</code>) and the species name (<code>\2</code>). Finally, the <code>p</code> at the end means "print the resulting line if the substitution was successful", so only lines matching the desired pattern are actually printed. </p>

CC BY-SA 3.0
2017-10-27T17:24:50.950Z
Thanks, that helps already. Is there a quick way to also extract the taxonomic kingdom and store it in the third column?
Cleb
CC BY-SA 3.0
2017-10-28T21:13:20.613Z
@Cleb sure, see updated answer. Next time, please explain all these details in your question (I mean tell us exactly what information you want to extract) so we can give you a working answer directly.
terdon
CC BY-SA 3.0
2017-10-27T16:04:58.860Z
2017-10-30T09:27:52.827Z
2017-10-30T09:27:52.827Z
terdon
<p>These are <a href="https://www.ncbi.nlm.nih.gov/taxonomy" rel="nofollow noreferrer">NCBI taxonomy</a> ID's rather than anything specific to UniProt. One way to parse them would be using BioPython via <a href="http://biopython.org/DIST/docs/api/Bio.Entrez-module.html" rel="nofollow noreferrer"><code>Bio.Entrez</code></a>, for example:</p>

<pre><code>from Bio import Entrez

def get_taxon_name_from_NCBI_id(id_):
    Entrez.email = 'name@provider.com'
    handle = Entrez.efetch(db='taxonomy', id= id_, retmode='xml')
    records = Entrez.read(handle)[0]
    return records['ScientificName'], records['TaxId']

print(get_taxon_name_from_NCBI_id(9606))
# ('Homo sapiens', '9606')
</code></pre>

<p>Of course you could query Entrez directly:
<a href="https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=taxonomy&amp;id=9606" rel="nofollow noreferrer">https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=taxonomy&amp;id=9606</a></p>

<p>Or even more basically do a manual search of NCBI taxonomy:
<a href="https://www.ncbi.nlm.nih.gov/taxonomy/?term=9606" rel="nofollow noreferrer">https://www.ncbi.nlm.nih.gov/taxonomy/?term=9606</a></p>

CC BY-SA 3.0
2017-10-27T17:23:41.560Z
I was not precise enough in my question: What I am after are the IDs, so basically what you use as input and there associated name and taxonomic kingdom. Ideally it would be a dictionary of the form `{org_id: {'name': 'org_name', 'kingdom': 'org_kingdom'}}` but I guess I can easily do this directly from NCBI; thanks for making me aware of this!
Cleb
CC BY-SA 3.0
2017-10-27T15:01:58.770Z
2017-10-27T15:20:02.983Z
2017-10-27T15:20:02.983Z
Chris_Rands
<p>I try to find organism IDs used by UniProt (e.g. 9606 for "Homo sapiens") and information about the 'taxonomic kingdom' this organism belongs to. The only source on UniProt I could find is <a href="http://www.uniprot.org/docs/speclist" rel="nofollow noreferrer">this file</a> which contains all required information. So in the Python world it would be a dictionary of the form:</p>

<pre><code>{org_id: {'name': 'org_name', 'kingdom': 'org_kingdom'}}
</code></pre>

<p>or a dataframe.</p>

<p>However, the file from above is not particular nice to parse due to its inconsistent column format (not impossible to parse though), so I am wondering whether there is an alternative source to retrieve this information or whether there is maybe even a webservice available that takes care of this parsing.</p>

<database><data-retrieval><uniprot><webservice>
How to access organism IDS, taxon nodes from UniProt?
# How to access organism IDS, taxon nodes from UniProt?

(Asked by: username_0 on 2017-10-27)

I try to find organism IDs used by UniProt (e.g. 9606 for "Homo sapiens") and information about the 'taxonomic kingdom' this organism belongs to. The only source on UniProt I could find is this file which contains all required information. So in the Python world it would be a dictionary of the form:
{org_id: {'name': 'org_name', 'kingdom': 'org_kingdom'}}

or a dataframe.
However, the file from above is not particular nice to parse due to its inconsistent column format (not impossible to parse though), so I am wondering whether there is an alternative source to retrieve this information or whether there is maybe even a webservice available that takes care of this parsing.

**Tags:** database, data-retrieval, uniprot, webservice

---

## 2 answers

### Answer by username_1 on 2017-10-27. Score: 1 | Accepted answer

That file is actually trivial to parse. You only care about lines that have N=$species, so you can simply do:
sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist 

That will return a tab separated list of taxID and species:
$ sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist | head
648330  Aedes albopictus densovirus (isolate Boublik/1994)
10804   Adeno-associated virus 2
648242  Adeno-associated virus 2 (isolate Srivastava/1982)
118452  Abacion magnum
72259   Abaeis nicippe
102642  Abax parallelepipedus
392897  Abalistes stellaris
75332   Abbottina rivularis
515833  Abdopus aculeatus
56673   Antarctic bacterium DS2-3R

So to get human, for example, you could do:
$ sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist | grep sapiens
63221   Homo sapiens neanderthalensis
9606    Homo sapiens

And, for exact matches:
$ sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist | 
    awk -F'\t' '$2=="Homo sapiens"'
9606    Homo sapiens

If you also want to keep the taxon node (kingdom), you can use:
sed -En 's/.* ([A-Z])  *([0-9]+): N=(.*)/\1\t\2\t\3/p' speclist 

Which prints:
$ sed -En 's/.* ([A-Z])  *([0-9]+): N=(.*)/\1\t\2\t\3/p' speclist  | head
V   648330  Aedes albopictus densovirus (isolate Boublik/1994)
V   10804   Adeno-associated virus 2
V   648242  Adeno-associated virus 2 (isolate Srivastava/1982)
E   118452  Abacion magnum
E   72259   Abaeis nicippe
E   102642  Abax parallelepipedus
E   392897  Abalistes stellaris
E   75332   Abbottina rivularis
E   515833  Abdopus aculeatus
B   56673   Antarctic bacterium DS2-3R


The sed command might appear a little daunting if you don't know sed, but it's actually quite simple. The -E flag enables extended regular expressions which give us a simpler syntax. The -n suppresses normal output, so that sed won't actually print anything unless explicitly told to. 
The s/from/to/ is the substitution operator and will replace from with to. Here, we are replacing zero or more characters (.*) until a space followed by one or more numbers ([0-9]+), a :, another space, then the string N= and then everything else (.*). The parentheses let us capture the matched strings, so that we can then refer to the 1st captured string as \1, the second as \2 etc. We are therefore replacing the entire line (because the regex matches the entire line) with the 1st pattern captured (the taxID, \1), a tab (\t) and the species name (\2). Finally, the p at the end means "print the resulting line if the substitution was successful", so only lines matching the desired pattern are actually printed.

> **Comments:**
> **username_0**: Thanks, that helps already. Is there a quick way to also extract the taxonomic kingdom and store it in the third column?  
> **username_1**: @Cleb sure, see updated answer. Next time, please explain all these details in your question (I mean tell us exactly what information you want to extract) so we can give you a working answer directly.  

---

### Answer by username_2 on 2017-10-27. Score: 4

These are NCBI taxonomy ID's rather than anything specific to UniProt. One way to parse them would be using BioPython via Bio.Entrez, for example:
from Bio import Entrez

def get_taxon_name_from_NCBI_id(id_):
    Entrez.email = 'name@provider.com'
    handle = Entrez.efetch(db='taxonomy', id= id_, retmode='xml')
    records = Entrez.read(handle)[0]
    return records['ScientificName'], records['TaxId']

print(get_taxon_name_from_NCBI_id(9606))
# ('Homo sapiens', '9606')

Of course you could query Entrez directly:
https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=taxonomy&id=9606
Or even more basically do a manual search of NCBI taxonomy:
https://www.ncbi.nlm.nih.gov/taxonomy/?term=9606

> **Comments:**
> **username_0**: I was not precise enough in my question: What I am after are the IDs, so basically what you use as input and there associated name and taxonomic kingdom. Ideally it would be a dictionary of the form `{org_id: {'name': 'org_name', 'kingdom': 'org_kingdom'}}` but I guess I can easily do this directly from NCBI; thanks for making me aware of this!  

---

