<p>In information theory notation, capital letters such as <span class="math-container">$X$</span> denote random variables, and lowercase letters such as <span class="math-container">$x$</span> mean their possible outcomes (i.e., fixed values). For example you can write the definition of expectation as <span class="math-container">$E[X] = \sum p(x) \cdot x$</span>. Moreover, vector values are denoted using length as superscripts, for example <span class="math-container">$X^n$</span> means <span class="math-container">$(X_1, \ldots, X_n)$</span> (similarly for <span class="math-container">$x^n$</span>). Moreover a substring <span class="math-container">$(X_i, \ldots, X_j)$</span> is often denoted by <span class="math-container">$X_i^j$</span>. These slides seem to be consistent with this convention. </p>

CC BY-SA 4.0
2020-04-05T18:54:02.127Z
2020-04-05T18:54:02.127Z
Mahdi Cheraghchi
<p>I have been reading whatever sources I could get my hands on today, regarding this problem. 
Most notes online about rate distortion theory come from the book <em>Elements of Information Theory</em> by Thomas M. Cover and Joy A. Thomas. The book seems well regarded so  I assume i am misunderstanding something.</p>

<p>I.e the following <a href="https://www2.isye.gatech.edu/~yxie77/ece587/Lecture20.pdf" rel="nofollow noreferrer">slides</a>. I am confused by the notation - which comes from the book. If we look at slides 14 and 16 in the slides deck linked. Firstly on slide 14 they denote a sequence of random variables (i.e a sequence of bits, as far as I understand) as <span class="math-container">$X^n$</span> 
<span class="math-container">$$
X^n = (X_{1}, \ldots, X_{n}), \quad X_{n} \sim p(x)
$$</span>
and the codeword for that sequence as <span class="math-container">$\hat{X}^n$</span>.  Now moving on to slide 16 they seem to mix between <span class="math-container">$X^n$</span> and <span class="math-container">$x^n$</span> i.e they mention that the distortion between sequences 
<span class="math-container">$$
d\left(x^{n}, \hat{x}^{n}\right)=\frac{1}{n} \sum_{i=1}^{n} d\left(x_{i}, \hat{x}_{i}\right)
$$</span></p>

<p>And distortion for a <span class="math-container">$\left(2^{n R}, n\right)$</span> code: 
<span class="math-container">$$
D=E d\left(X^{n}, g_{n}\left(f_{n}\left(X^{n}\right)\right)\right)=\sum_{x^{n}} p\left(x^{n}\right) d\left(x^{n}, g_{n}\left(f_{n}\left(x^{n}\right)\right)\right)
$$</span></p>

<p>but as far as i can <span class="math-container">$x^n$</span> and <span class="math-container">$X^n$</span>, and consequently <span class="math-container">$\hat{x}^n$</span> and <span class="math-container">$\hat{X}^n$</span> denote the same thing? What is the idea behind this, if any?  It would make more sense to me to keep the capitalization consistent unless I am missing something? This notation is consistent across all sources I could find.</p>

CC BY-SA 4.0
2020-03-26T15:54:21.527Z
To the person downvoting this. Do you mind mentioning why, so i can update the question?
sn3jd3r
CC BY-SA 4.0
2020-03-26T16:16:59.970Z
I didn't downvote, but your question is unclear. What is the same thing as what? And why do you think they should be different?
Peter Shor 
CC BY-SA 4.0
2020-03-26T16:19:06.997Z
@PeterShor Thank you. I can see that. I will update now.
sn3jd3r
CC BY-SA 4.0
2020-03-26T20:56:04.920Z
Typically, people use different notation (e.g., uppercase letters v. lowercase ones) when things are random variables (as opposed to deterministic or general, non necessarily random things). That seems to be what is happening here.
Clement C.
CC BY-SA 4.0
2020-03-27T22:22:38.413Z
The capital letters are the random variables. The lower-case letters are the possible values of the random variables.  So we have the equation $\mathrm{E} X = \sum_{x} p(x) x$. If you used all capital $X$'s in that equation, it would be confusing.
Peter Shor 
<it.information-theory><coding-theory><st.statistics>
Notation of sequences in rate distortion theory
sn3jd3r
# Notation of sequences in rate distortion theory

(Asked by: username_0 on 2020-03-26)

I have been reading whatever sources I could get my hands on today, regarding this problem.
Most notes online about rate distortion theory come from the book *Elements of Information Theory* by Thomas M. Cover and Joy A. Thomas. The book seems well regarded so I assume i am misunderstanding something.

I.e the following [slides](https://www2.isye.gatech.edu/~yxie77/ece587/Lecture20.pdf). I am confused by the notation - which comes from the book. If we look at slides 14 and 16 in the slides deck linked. Firstly on slide 14 they denote a sequence of random variables (i.e a sequence of bits, as far as I understand) as $X^n$
$$
X^n = (X\_{1}, \ldots, X\_{n}), \quad X\_{n} \sim p(x)
$$
and the codeword for that sequence as $\hat{X}^n$. Now moving on to slide 16 they seem to mix between $X^n$ and $x^n$ i.e they mention that the distortion between sequences
$$
d\left(x^{n}, \hat{x}^{n}\right)=\frac{1}{n} \sum\_{i=1}^{n} d\left(x\_{i}, \hat{x}\_{i}\right)
$$

And distortion for a $\left(2^{n R}, n\right)$ code:
$$
D=E d\left(X^{n}, g\_{n}\left(f\_{n}\left(X^{n}\right)\right)\right)=\sum\_{x^{n}} p\left(x^{n}\right) d\left(x^{n}, g\_{n}\left(f\_{n}\left(x^{n}\right)\right)\right)
$$

but as far as i can $x^n$ and $X^n$, and consequently $\hat{x}^n$ and $\hat{X}^n$ denote the same thing? What is the idea behind this, if any? It would make more sense to me to keep the capitalization consistent unless I am missing something? This notation is consistent across all sources I could find.

**Tags:** it.information-theory, coding-theory, st.statistics
> **Comments:**
> **username_0**: To the person downvoting this. Do you mind mentioning why, so i can update the question?  
> **username_1**: I didn't downvote, but your question is unclear. What is the same thing as what? And why do you think they should be different?  
> **username_0**: @PeterShor Thank you. I can see that. I will update now.  
> **username_2**: Typically, people use different notation (e.g., uppercase letters v. lowercase ones) when things are random variables (as opposed to deterministic or general, non necessarily random things). That seems to be what is happening here.  
> **username_1**: The capital letters are the random variables. The lower-case letters are the possible values of the random variables.  So we have the equation $\mathrm{E} X = \sum_{x} p(x) x$. If you used all capital $X$'s in that equation, it would be confusing.  

---

## 1 answers

### Answer by username_3 on 2020-04-05. Score: 1 | Accepted answer

In information theory notation, capital letters such as $X$ denote random variables, and lowercase letters such as $x$ mean their possible outcomes (i.e., fixed values). For example you can write the definition of expectation as $E[X] = \sum p(x) \cdot x$. Moreover, vector values are denoted using length as superscripts, for example $X^n$ means $(X\_1, \ldots, X\_n)$ (similarly for $x^n$). Moreover a substring $(X\_i, \ldots, X\_j)$ is often denoted by $X\_i^j$. These slides seem to be consistent with this convention.


---

