<p>I am not sure there is a state of the art for that.</p>
<p>Some possible strategies I can think of:</p>
<ul>
<li>Applying a dictionary to identify language words and discard these.</li>
<li>Using some context information like identifying some words in the same sentence than the sequence.</li>
<li>I guess the format in papers for sequences will not be the same as for text. Maybe you can find a pattern.</li>
</ul>
<p>I would use an incremental strategy. Use the simplest algorithm then see what it recovers. If there is garbage, find a strategy to remove some garbage with a common pattern. Repeat until you are happy with the result.</p>
<p>Disclaimer: I have some experience in Natural Language Processing.</p>

CC BY-SA 4.0
2020-10-26T19:35:17.060Z
2020-10-26T19:35:17.060Z
juanjo75es
<p>The text mining literature has an emphasis on identifying and normalizing gene names, mutations, pathways, concepts, and so on. I haven't been able to find much, however, on methods for extracting nucleotide and peptide sequences from documents. I did find this tidbit from <a href="https://doi.org/10.1186/gb-2008-9-2-r31" rel="nofollow noreferrer">Aerts <em>et al.</em> (2008)</a>, emphasis mine:</p>
<blockquote>
<p>Text was split into words and words greater than 10 characters in length with greater than 40% of characters from the capitalized DNA alphabet [ACGT] were extracted using regular expressions to isolate putative DNA sequences.
All putative DNA sequences extracted from each paper were concatenated in the order they appeared in the text into a single fasta sequence and labeled with the corresponding PMID.
Concatenation of sequences was performed to merge sequences split by line breaks in the text conversion, and <strong>because we reasoned that inappropriate joins would be reconciled at the genome level by local alignment procedures</strong>.
Extracted, concatenated sequences were used as queries to BLAST RepeatMasked versions of genome sequences downloaded from the UCSC genome database...</p>
</blockquote>
<p>I'm struck by how simple the author's method for extracting DNA sequences is. But I guess with the limited complexity of the DNA alphabet, that makes sense. This approach wouldn't work with the much more complex protein alphabet.</p>
<p>Are simple regular expressions the text mining community's state of the art for finding DNA/RNA sequences in documents? What methods are used to identify protein sequences?</p>

<proteins><phylogenetics><text-mining>
Mining documents for nucleotide and peptide sequences
# Mining documents for nucleotide and peptide sequences

(Asked by: username_0 on 2020-10-26)

The text mining literature has an emphasis on identifying and normalizing gene names, mutations, pathways, concepts, and so on. I haven't been able to find much, however, on methods for extracting nucleotide and peptide sequences from documents. I did find this tidbit from [Aerts *et al.* (2008)](https://doi.org/10.1186/gb-2008-9-2-r31), emphasis mine:

> Text was split into words and words greater than 10 characters in length with greater than 40% of characters from the capitalized DNA alphabet [ACGT] were extracted using regular expressions to isolate putative DNA sequences.
> All putative DNA sequences extracted from each paper were concatenated in the order they appeared in the text into a single fasta sequence and labeled with the corresponding PMID.
> Concatenation of sequences was performed to merge sequences split by line breaks in the text conversion, and **because we reasoned that inappropriate joins would be reconciled at the genome level by local alignment procedures**.
> Extracted, concatenated sequences were used as queries to BLAST RepeatMasked versions of genome sequences downloaded from the UCSC genome database...

I'm struck by how simple the author's method for extracting DNA sequences is. But I guess with the limited complexity of the DNA alphabet, that makes sense. This approach wouldn't work with the much more complex protein alphabet.

Are simple regular expressions the text mining community's state of the art for finding DNA/RNA sequences in documents? What methods are used to identify protein sequences?

**Tags:** proteins, phylogenetics, text-mining

---

## 1 answers

### Answer by username_1 on 2020-10-26. Score: 2

I am not sure there is a state of the art for that.

Some possible strategies I can think of:

* Applying a dictionary to identify language words and discard these.
* Using some context information like identifying some words in the same sentence than the sequence.
* I guess the format in papers for sequences will not be the same as for text. Maybe you can find a pattern.

I would use an incremental strategy. Use the simplest algorithm then see what it recovers. If there is garbage, find a strategy to remove some garbage with a common pattern. Repeat until you are happy with the result.

Disclaimer: I have some experience in Natural Language Processing.


---

