<p>Why do you think the entropy of 0 is incorrect? It intuitively makes sense, as there is no uncertainty about the base at position 3, and thus there is no entropy.</p>

<p>However, what is plotted in a sequence logo isn't the entropy, but rather a measure of the "decrease in uncertainty" as the sequence is aligned. This is calculated by taking the entropy at this position if we randomly aligned sequences (<span class="math-container">$H_g(L)$</span>), and subtracting from it the entropy of the alignment (<span class="math-container">$H_g(s)$</span>): 
<span class="math-container">$$
R_{sequence}=H_g(L) - H_s(L)
$$</span></p>

<p>The entropy at position 3 based on a random alignment is calculated by assuming there are 4 equally likely events (one for each base), and thus:
<span class="math-container">$$
\begin{align*} 
H_g(L) &amp; = -((1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2)) \\
H_g(L) &amp; = 2
\end{align*}
$$</span>
Notably, <span class="math-container">$H_g(L)$</span> will always be 2 when dealing with nucleotide sequences.</p>

<p>So in your example, we have:
<span class="math-container">$$
\begin{align*} 
R_{sequence}&amp;=H_g(L) - H_s(L) \\
R_{sequence}&amp;=2 - 0 \\
R_{sequence}&amp;=2 \text{ bits of entropy}
\end{align*}
$$</span></p>

<p>Finally, to work out the height of each base at each position in the logo, we multiply the frequency of that base by the overall information gain <span class="math-container">$R_{sequence}$</span>. In this case the frequency of the base A at position 3 is of course 1:
<span class="math-container">$$
\begin {align*}
H(b,l) &amp;=f(b, l) \times R_{sequence} \\
H(A, 3) &amp;= f(a, 3) \times 2 \\
&amp;= 1 \times 2 \\
&amp;= 2
\end{align*}
$$</span></p>

<p>Source: <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC332411/" rel="nofollow noreferrer">Sequence logos: a new way to display consensus sequences.
 (Thomas D. Schneider, R. Michael Stephens, 1990)</a></p>

<p>Note: I've excluded the correction factor <span class="math-container">$e(n)$</span> from the overall information-gain calculation for the sake of simplicity. Refer to the paper above for an explanation of this.</p>

CC BY-SA 4.0
2019-07-30T22:33:26.880Z
Thanks for your answer. What does "g" mean/represent/come from in $H_g(L)$
czlsws
CC BY-SA 4.0
2019-07-31T03:00:43.367Z
I'm not sure. My guess is g=genome and s=sequence(ing) but that's pure conjecture since the paper doesn't spell it out
Migwell
CC BY-SA 4.0
2019-07-31T03:47:27.403Z
Thanks for your reply. And does L mean/represent/come from `location` since the paper says "at various positions (L)".
czlsws
CC BY-SA 4.0
2019-07-31T03:57:14.310Z
Yes, that would be my assumption. This is important to include in the equation because it's only calculating the information at a single position. The authors propose another equation for an entire sequence (which is just summing these per-position scores)
Migwell
CC BY-SA 4.0
2019-07-31T13:35:18.993Z
Thanks a lot. And last question, does R come from/represent/mean acids?
czlsws
CC BY-SA 4.0
2019-07-31T14:56:51.003Z
I don't know. But also I think you're putting too much importance on the authors' choice of variables. You could write any of these equations with different letters and they would be just as correct.
Migwell
CC BY-SA 4.0
2019-07-30T13:10:52.197Z
2019-07-31T15:12:19.477Z
2019-07-31T15:12:19.477Z
Migwell
<p>I'm confused by the <a href="https://bioinformatics.stackexchange.com/q/9091/4962">computation</a> of sequence logo. <a href="https://en.wikipedia.org/wiki/Sequence_logo#Logo_creation" rel="nofollow noreferrer">Wikipedia</a> gives a process about this without a concrete example.</p>

<p>Let's just consider DNA, so there are 4 bases (nucleic acids).</p>

<p>The following data comes from the book "Machine Learning - A Probabilistic Perspective (Figure 1)"</p>

<p><a href="https://i.sstatic.net/XP7q1.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/XP7q1.png" alt="enter image description here"></a></p>

<p>The Shannon entropy of position <span class="math-container">$i$</span> is:</p>

<p><span class="math-container">${\displaystyle H_{i}=-\sum _{b=a}^{t}f_{b,i}\times \log _{2}f_{b,i}}$</span></p>

<p>Where <span class="math-container">${\displaystyle f_{b,i}}$</span> is the relative frequency of base</p>

<p>This <a href="https://bioinformatics.stackexchange.com/a/9094/4962">post</a> is computing position 3, where it seems that the relative frequency of <code>a</code> is 100%, and the relative frequency of 3 other bases are 0%.</p>

<p>Now, how do you calculate the Shannon entropy of position 3? It seems to be 0, which is apparently incorrect, where am I wrong?</p>

CC BY-SA 4.0
2019-07-30T06:48:19.637Z
Note that the frequency is always in the interval [0,1] instead of [0,100]. H is 0 for columns with only one symbol.
Peter Menzel
<phylogenetics><ngs><sequence-analysis>
How to compute the Shannon entropy for a strand of DNA?
# How to compute the Shannon entropy for a strand of DNA?

(Asked by: username_0 on 2019-07-30)

I'm confused by the [computation](https://bioinformatics.stackexchange.com/q/9091/4962) of sequence logo. [Wikipedia](https://en.wikipedia.org/wiki/Sequence_logo#Logo_creation) gives a process about this without a concrete example.

Let's just consider DNA, so there are 4 bases (nucleic acids).

The following data comes from the book "Machine Learning - A Probabilistic Perspective (Figure 1)"

[![enter image description here](https://i.sstatic.net/XP7q1.png)](https://i.sstatic.net/XP7q1.png)

The Shannon entropy of position $i$ is:

${\displaystyle H\_{i}=-\sum \_{b=a}^{t}f\_{b,i}\times \log \_{2}f\_{b,i}}$

Where ${\displaystyle f\_{b,i}}$ is the relative frequency of base

This [post](https://bioinformatics.stackexchange.com/a/9094/4962) is computing position 3, where it seems that the relative frequency of `a` is 100%, and the relative frequency of 3 other bases are 0%.

Now, how do you calculate the Shannon entropy of position 3? It seems to be 0, which is apparently incorrect, where am I wrong?

**Tags:** phylogenetics, ngs, sequence-analysis
> **Comments:**
> **username_1**: Note that the frequency is always in the interval [0,1] instead of [0,100]. H is 0 for columns with only one symbol.  

---

## 1 answers

### Answer by username_2 on 2019-07-30. Score: 5 | Accepted answer

Why do you think the entropy of 0 is incorrect? It intuitively makes sense, as there is no uncertainty about the base at position 3, and thus there is no entropy.

However, what is plotted in a sequence logo isn't the entropy, but rather a measure of the "decrease in uncertainty" as the sequence is aligned. This is calculated by taking the entropy at this position if we randomly aligned sequences ($H\_g(L)$), and subtracting from it the entropy of the alignment ($H\_g(s)$):
$$
R\_{sequence}=H\_g(L) - H\_s(L)
$$

The entropy at position 3 based on a random alignment is calculated by assuming there are 4 equally likely events (one for each base), and thus:
$$
\begin{align\*}
H\_g(L) & = -((1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2)) \\
H\_g(L) & = 2
\end{align\*}
$$
Notably, $H\_g(L)$ will always be 2 when dealing with nucleotide sequences.

So in your example, we have:
$$
\begin{align\*}
R\_{sequence}&=H\_g(L) - H\_s(L) \\
R\_{sequence}&=2 - 0 \\
R\_{sequence}&=2 \text{ bits of entropy}
\end{align\*}
$$

Finally, to work out the height of each base at each position in the logo, we multiply the frequency of that base by the overall information gain $R\_{sequence}$. In this case the frequency of the base A at position 3 is of course 1:
$$
\begin {align\*}
H(b,l) &=f(b, l) \times R\_{sequence} \\
H(A, 3) &= f(a, 3) \times 2 \\
&= 1 \times 2 \\
&= 2
\end{align\*}
$$

Source: [Sequence logos: a new way to display consensus sequences.
(Thomas D. Schneider, R. Michael Stephens, 1990)](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC332411/)

Note: I've excluded the correction factor $e(n)$ from the overall information-gain calculation for the sake of simplicity. Refer to the paper above for an explanation of this.

> **Comments:**
> **username_0**: Thanks for your answer. What does "g" mean/represent/come from in $H_g(L)$  
> **username_2**: I'm not sure. My guess is g=genome and s=sequence(ing) but that's pure conjecture since the paper doesn't spell it out  
> **username_0**: Thanks for your reply. And does L mean/represent/come from `location` since the paper says "at various positions (L)".  
> **username_2**: Yes, that would be my assumption. This is important to include in the equation because it's only calculating the information at a single position. The authors propose another equation for an entire sequence (which is just summing these per-position scores)  
> **username_0**: Thanks a lot. And last question, does R come from/represent/mean acids?  
> **username_2**: I don't know. But also I think you're putting too much importance on the authors' choice of variables. You could write any of these equations with different letters and they would be just as correct.  

---

