<p><span class="math-container">$S \to X \to Y \to \hat{S}$</span></p>
<p><span class="math-container">$\text{source} \to \text{input} \to \text{output} \to \text{target}$</span></p>
<p>In information theory introductory books, an encoder is usually defined as a deterministic function <span class="math-container">$f:\mathcal{A}_\text{S} \to \mathcal{A}_\text{X}$</span> that converts a symbol from the alphabet of the source into a code (a symbol of the input of the channel). But it is not uncommon to see it defined as a stochastic map p(X|S). The channel is normally denoted as p(Y|X). In practice, is there any difference between an encoder and a channel? They are both transformations after all.</p>
<ul>
<li>changed the title to reflect better what I mean.</li>
</ul>

CC BY-SA 4.0
2020-11-13T01:49:34.253Z
Could you explain what you mean by `In practice, is there any difference between an encoder and a channel?`
droptop
CC BY-SA 4.0
2020-11-13T12:19:28.587Z
What I mean is that if an encoder is P(X|S) and a channel is P(Y|X) they both represent a stochastic map between 2 random variables. Therefore, it seems to me that you can apply Shannon`s noisy channel to P(X|S)
Fred Guth
CC BY-SA 4.0
2020-11-14T18:31:26.940Z
How would you define the rate, $R$ between the source and the input??
droptop
CC BY-SA 4.0
2020-11-15T01:47:34.020Z
as the compression limit of the encoder P(X|S).
Fred Guth
CC BY-SA 4.0
2020-11-16T12:10:01.433Z
Wouldn't that instead be the capacity??
droptop
CC BY-SA 4.0
2020-11-16T12:42:48.390Z
lossy compression ( in the case of $P(X|S)$ ) is analogous to channel noise and compression limit analogous to channel capacity which depends on the randomness of $S$.
droptop
CC BY-SA 4.0
2020-11-17T02:00:49.963Z
Wonderful! That is exactly what I though. I keep seeing Deep Learning papers where they use$Y \to X \to Z \to \hat{Y}$ and call p(Z|X) an encoder... which I agree in the sense that it is a code transformation: encoder - encoder - decoder, all stochastic. So, I guess the difference is that a channel is one that "nature" or something else changes the code, and lossy encoder is when it is a transformation that we acted upon.
Fred Guth
CC BY-SA 4.0
2020-11-17T02:06:25.260Z
Maybe I should have asked the difference between a lossy encoder and a noisy channel. Which seems to be the same thing but mean different things in their applications.
Fred Guth
<it.information-theory>
Difference between a lossy encoder and a noisy channel in Information Theory
Fred Guth
# Difference between a lossy encoder and a noisy channel in Information Theory

(Asked by: username_0 on 2020-11-13)

$S \to X \to Y \to \hat{S}$

$\text{source} \to \text{input} \to \text{output} \to \text{target}$

In information theory introductory books, an encoder is usually defined as a deterministic function $f:\mathcal{A}\_\text{S} \to \mathcal{A}\_\text{X}$ that converts a symbol from the alphabet of the source into a code (a symbol of the input of the channel). But it is not uncommon to see it defined as a stochastic map p(X|S). The channel is normally denoted as p(Y|X). In practice, is there any difference between an encoder and a channel? They are both transformations after all.

* changed the title to reflect better what I mean.

**Tags:** it.information-theory
> **Comments:**
> **username_1**: Could you explain what you mean by `In practice, is there any difference between an encoder and a channel?`  
> **username_0**: What I mean is that if an encoder is P(X|S) and a channel is P(Y|X) they both represent a stochastic map between 2 random variables. Therefore, it seems to me that you can apply Shannon`s noisy channel to P(X|S)  
> **username_1**: How would you define the rate, $R$ between the source and the input??  
> **username_0**: as the compression limit of the encoder P(X|S).  
> **username_1**: Wouldn't that instead be the capacity??  
> **username_1**: lossy compression ( in the case of $P(X|S)$ ) is analogous to channel noise and compression limit analogous to channel capacity which depends on the randomness of $S$.  
> **username_0**: Wonderful! That is exactly what I though. I keep seeing Deep Learning papers where they use$Y \to X \to Z \to \hat{Y}$ and call p(Z|X) an encoder... which I agree in the sense that it is a code transformation: encoder - encoder - decoder, all stochastic. So, I guess the difference is that a channel is one that "nature" or something else changes the code, and lossy encoder is when it is a transformation that we acted upon.  
> **username_0**: Maybe I should have asked the difference between a lossy encoder and a noisy channel. Which seems to be the same thing but mean different things in their applications.  

---

## 0 answers

