<p>This is the 'guess' of the second decoder about what would be the a priori information in the above formula (to be used by the other first decoder).
And this exactly message passing of 'extrinsic' information is the major contribution of the French researchers.</p>

<p>Bytheway your formula has typo error, dominator/numerators are in the opossite way, please correct.</p>

CC BY-SA 3.0
2016-11-07T14:34:58.833Z
2016-11-07T14:34:58.833Z
Anemos
<p>I'm self-studying turbo codes for a graduate course in coding theory. I understood how turbo codes works by directly reading Berrou' paper and some of the following works on this topic. Given that, there is a simple question that i canno't answer with my self...
Turbo code relies on the fundamental principle of message passing during iterative decoding, more formally speaking the extrinsic information produced by one decoder is passed as a-priori information to the next (companion) decoder. This is my problematic point: why extrinsic information, that is an a-posteriori computation (given from the Log-likelihood-ratio) should be passed as an a-priori information the next decoder?</p>

<p>The LLR can be expressed as:
$$L(u_i)=\log\frac{P(u_i=1|\text{observation})}{P(u_i=0|\text{observation})}=\log\frac{p(obs.|u_i=1)}{p(obs.|u_i=0)}+\log\frac{P(u_i=0)}{P(u_i=1)}$$</p>

<p>What if the apriori knowledge of the source is known? For example...suppose a indipendent binary source, so that $P(0)=P(1)=1/2$. In this case there is no need to update the apriori knowledge because it is exactly equal to $0$, and extrinsic information is, in general, different from $0$, so the estimate is wrong at each step.</p>

<p>Thanks in advance</p>

CC BY-SA 3.0
2016-01-02T23:13:27.793Z
The bits of the codeword don't have the same probabilities as the bits of the source. Because you encode the source to get the codeword. What you're doing is looking at the ***conditional*** probability of the bits of the ***codeword***, given what you already know.
Peter Shor 
CC BY-SA 3.0
2016-01-03T11:16:39.067Z
That's ok but..in the LLR you're searching for the "best encoder input" given observation, and then, by applying Bayes' theorem, you obtain the rule i wrote in my question. $u_i$ in this case are the encoder input, not the codeword, so i can assume to know the statistical characterization of my source (i.e. the second term of my summation). I'm i wrong somewhere?
steg
CC BY-SA 3.0
2016-01-03T13:43:07.567Z
A turbo code contains two convolutional codes. in the $k$'th step, the first term only considers one of the convolutional codes, while you take the other convolutional code into account by treating the knowledge you obtained by looking at it as *a priori* information.
Peter Shor 
<it.information-theory><message-passing>
Turbo codes and message passing
steg
# Turbo codes and message passing

(Asked by: username_0 on 2016-01-02)

I'm self-studying turbo codes for a graduate course in coding theory. I understood how turbo codes works by directly reading Berrou' paper and some of the following works on this topic. Given that, there is a simple question that i canno't answer with my self...
Turbo code relies on the fundamental principle of message passing during iterative decoding, more formally speaking the extrinsic information produced by one decoder is passed as a-priori information to the next (companion) decoder. This is my problematic point: why extrinsic information, that is an a-posteriori computation (given from the Log-likelihood-ratio) should be passed as an a-priori information the next decoder?

The LLR can be expressed as:
$$L(u\_i)=\log\frac{P(u\_i=1|\text{observation})}{P(u\_i=0|\text{observation})}=\log\frac{p(obs.|u\_i=1)}{p(obs.|u\_i=0)}+\log\frac{P(u\_i=0)}{P(u\_i=1)}$$

What if the apriori knowledge of the source is known? For example...suppose a indipendent binary source, so that $P(0)=P(1)=1/2$. In this case there is no need to update the apriori knowledge because it is exactly equal to $0$, and extrinsic information is, in general, different from $0$, so the estimate is wrong at each step.

Thanks in advance

**Tags:** it.information-theory, message-passing
> **Comments:**
> **username_1**: The bits of the codeword don't have the same probabilities as the bits of the source. Because you encode the source to get the codeword. What you're doing is looking at the ***conditional*** probability of the bits of the ***codeword***, given what you already know.  
> **username_0**: That's ok but..in the LLR you're searching for the "best encoder input" given observation, and then, by applying Bayes' theorem, you obtain the rule i wrote in my question. $u_i$ in this case are the encoder input, not the codeword, so i can assume to know the statistical characterization of my source (i.e. the second term of my summation). I'm i wrong somewhere?  
> **username_1**: A turbo code contains two convolutional codes. in the $k$'th step, the first term only considers one of the convolutional codes, while you take the other convolutional code into account by treating the knowledge you obtained by looking at it as *a priori* information.  

---

## 1 answers

### Answer by username_2 on 2016-11-07. Score: 1

This is the 'guess' of the second decoder about what would be the a priori information in the above formula (to be used by the other first decoder).
And this exactly message passing of 'extrinsic' information is the major contribution of the French researchers.

Bytheway your formula has typo error, dominator/numerators are in the opossite way, please correct.


---

