<p>When the $X_i$ are iid for some distribution with non-negative essential support (<em>i.e.</em>, the probability of a negative value is zero) and the distribution of $\log(X)$ has a finite variance, then the Central Limit Theorem easily implies that $X^* = \log(X_1 \times \cdots \times X_k)$ approximately has a normal (Gaussian) distribution for sufficiently large $k$.</p>

<p>The problem here is that each $X_i$ has a 50% chance of being negative, precluding (easy) analysis with logarithms.  Nevertheless, only two things can happen (almost surely, after neglecting the zero chance that any of the $X_i$ equals $0$): either an even number of the $X_i$ are negative, in which case the product is positive, or an odd number of the $X_i$ are negative, in which case the product is negative.  Each case has a 50% chance of happening.  Whence, the probability density of the product has two parts (symmetric to each other about 0) and each part looks like the product of draws from a <a href="http://en.wikipedia.org/wiki/Half-normal_distribution" rel="nofollow noreferrer">half-normal distribution</a>.  A calculation shows that the log of a half-normal distribution has finite variance, allowing us to apply the preceding observation and conclude that for sufficiently large $k$, the distribution of $\log(X^*)$ <em>conditional</em> on $X^* \gt 0$ is approximately Gaussian and the distribution of $\log(-X^*)$ conditional on $X^* \lt 0$ is exactly the same.  ($X^*$ is an equal mixture of variables $Y$ and $Z$ where $Y$ is approximately lognormal and $-Z$ has the same distribution as $Y$.) </p>

<p>In effect, the distribution of $X^*$ has two pieces: the positive piece is approximately lognormal and the negative piece is its reflection about 0.  This makes it bimodal; all odd moments are $0$; and small even moments can be approximated from the Central Limit Theorem.  In particular, the mean of the positive part of $\log(X^*)$ equals $-(\log(2) + \gamma)k/2$, approximately $-0.635181 k$, and its variance equals $\pi^2 k/8$, approximately $1.2337 k$.</p>

<p>Simulations indicate this approximation is good for $k \gt 100$, more or less.  Considerable negative skewness in the distribution of $\log(|X^*|)$ is evident for smaller $k$.</p>

<p><img src="https://i.sstatic.net/3lF8i.png" alt="Histogram for k=100"></p>

<p><em>This (unnormalized) histogram documents 100,000 independent realizations of $\log(|X^*|)$ with $k=100$.  Very slight negative skewness is still visible, but the general Gaussian shape is apparent, centered as expected around -63.5 with variance 123.3 (standard deviation 11.1).</em></p>

<p>It is evident from the wide range of highly negative logarithms that an attempt to plot the distribution of $X^*$ itself would be useless: it would show as a unit spike at 0.  If we normalize it, though, a plot is informative.  Specifically, consider the "signed geometric mean" $\text{Sign}(X^*)\exp(\log(|X^*|)/k)$:</p>

<p><img src="https://i.sstatic.net/2X6Rq.png" alt="Histogram of the signed geometric mean"></p>

<p><em>This histogram displays the same data as the previous one, but normalized as signed geometric means.  The shape on the right is a lognormal distribution; the shape on the left is its reflection about 0.</em></p>

CC BY-SA 3.0
2011-08-17T19:57:38.413Z
2011-08-18T20:41:25.413Z
2011-08-18T20:41:25.413Z
whuber
<p>The distribution of the multiplication of independent Gaussians is not close to a Gaussian.</p>

<p>There are various ways to convince yourself in this. E.g., compute the expectation and the variance of the product from the expectation and the variance of the individual variables.
This gives you a candidate Gaussian if there were a Gaussian approximation.
Now, think of some simple properties of the product (e.g., the probability it is 0) and see if your candidate is good enough as an approximation.</p>

CC BY-SA 3.0
2011-08-13T23:02:40.267Z
Maybe that's true, since the 4th and higher moments don't match and the discrepancy becomes greater as k grows. But is this distribution studied somewhere, say for k=2 ?
Mahdi Cheraghchi
CC BY-SA 3.0
2011-08-14T01:49:19.950Z
Can whoever down-voted my answer please provide an explanation for why they did so?
Dana Moshkovitz
CC BY-SA 3.0
2011-08-14T02:35:13.857Z
MCH, I trust that you saw that Jukka answered your question above.
Dana Moshkovitz
CC BY-SA 3.0
2011-08-14T03:12:20.300Z
Yes, indeed the link answers the question for k=2, 3. It's interesting to see that the functions involved get increasingly complex even for very small k. This seems inevitable if one wishes to find out the exact distribution. It would be nice though to have some quantitative estimates about the distribution, I'll dig up the literature to see if something's already known. Thanks!
Mahdi Cheraghchi
CC BY-SA 3.0
2011-08-14T05:13:16.550Z
Please explain before giving downvotes. I agree with Dana's comment, and am also puzzled why the answer would get a downvote.
Suresh Venkat
CC BY-SA 3.0
2011-08-14T07:51:40.860Z
@Suresh I am not the downvoter, but I did consider it. The reason I considered it is because I think the question if off-topic and from meta discussions like [this](http://meta.cstheory.stackexchange.com/q/601/1037) I got the impression that we should discourage answering off-topic questions.
Artem Kaznatcheev
CC BY-SA 3.0
2011-08-14T21:34:40.973Z
My interpretation is that down-voting an answer is like flagging it false, misleading, irrelevant etc. If you don't want to encourage something don't up-vote it.
Dana Moshkovitz
CC BY-SA 3.0
2011-08-14T22:38:51.157Z
Or down-voting can mean “this should not have been posted.”  People interpret the same thing differently, and you cannot tell others when to down-vote answers and when not to.
Tsuyoshi Ito
CC BY-SA 3.0
2011-08-17T19:59:06.087Z
I agree that this question belongs elsewhere.  First choice should be stats, because it's a natural one for statisticians (and it has been tagged as statistical), but math would be ok too.
whuber
CC BY-SA 3.0
2011-08-17T23:50:31.547Z
@whuber: I understand that you are a high rep user and moderator on stats.SE and of course this might be more suitable for stats.SE, but it is also fine here, mathematical questions related to CS are in inside the scope of cstheory, this is not out of scope on cstheory. Often questions get cross-posted on cstheory and MO and that is fine.
Kaveh
CC BY-SA 3.0
2011-08-17T23:56:04.907Z
@Artem, I don't find it reasonable to down vote an answer for answering a question which is not *clearly* off-topic, this does not look like a homework question to me, in any case, I think there was an agreement that it is not OK to down vote without commenting and explaining the reason. @ all, it would be more reasonable to continue on meta if there are disagreements and you want to continue discussing them.
Kaveh
CC BY-SA 3.0
2011-08-18T00:00:46.827Z
@Kaveh: No matter what you say, some people down-vote without stating reasons or for reasons you do not find reasonable.  You have to accept the plain fact that some people act in a way which you find unreasonable.  Also, I had thought that the question was obviously off topic until I saw that a wrong answer was posted (which was deleted afterward).  Honestly speaking, I still think that the question is off topic, but I do not want to waste energy arguing that.
Tsuyoshi Ito
CC BY-SA 3.0
2011-08-18T00:05:07.390Z
@Tsuyoshi, you are correct about down votes, I was just explaining the norms and agreements (no one argued against it in the Town Hall meeting AFAIR), of course people can violate them if they want, as we see in this case, and we can complain about those violations, which is what we are doing here.
Kaveh
CC BY-SA 3.0
2011-08-18T00:17:55.053Z
@Kaveh I have started a meta discussion about [this already](http://meta.cstheory.stackexchange.com/questions/1264/conventions-on-downvoting). Note however, that I did not downvote this answer, I just considered doing so when I thought the question was off-topic. Let's discuss more on the linked meta question.
Artem Kaznatcheev
CC BY-SA 3.0
2011-08-18T01:32:30.897Z
@Kaveh That's ok.  I just didn't recognize any explicit CS application in this question.
whuber
CC BY-SA 3.0
2011-08-13T22:48:16.370Z
2011-08-14T02:33:33.280Z
2011-08-14T02:33:33.280Z
Dana Moshkovitz
<p>Suppose X_1, ..., X_k are iid standard Gaussian variables, for some k > 1.</p>

<p>Then, what is the distribution of X := X_1 * ... * X_k ? Can it be approximated by a Gaussian, maybe for large k ?</p>

CC BY-SA 3.0
2011-08-13T23:59:34.003Z
I am failing to see how this questions is related to theoretical computer science. If this is only statistics/probability you can ask on http://stats.stackexchange.com/.
Mohammad Alaggan
CC BY-SA 3.0
2011-08-14T00:32:55.820Z
The FAQ of this site: http://cstheory.stackexchange.com/faq.
Mohammad Alaggan
CC BY-SA 3.0
2011-08-14T00:49:20.623Z
I agree that this is off-topic here. But this page might be useful (covers some special cases):  http://mathworld.wolfram.com/NormalProductDistribution.html
Jukka Suomela
CC BY-SA 3.0
2011-08-14T01:00:45.363Z
Polynomials of Gaussian random variables arise in theoretical CS often, especially in Fourier analysis of Boolean functions, inapproximability and so on. I'm asking here since I believe TCS researchers may have more "CS-friendly" answers to this question (e.g., involving approximations, quantitative bounds, etc).
Mahdi Cheraghchi
CC BY-SA 3.0
2011-08-14T03:04:39.623Z
I agree, MCH. I don't understand the objections.
arnab
CC BY-SA 3.0
2011-08-14T03:49:46.103Z
Although MO or stats.SE might be a better place to ask this question, I don't think it is off-topic for cstheory, approximating distributions is also studied in TCS.
Kaveh
CC BY-SA 3.0
2011-08-14T17:08:04.703Z
@MCH: Perhaps you could edit the question to emphasise TCS connections and TCS aspects, to make it more obvious that it is on-topic here? And now that we have already some answers – and it seems that the general case is very complicated – perhaps you could also tell us more about the specific requirements of your application?
Jukka Suomela
CC BY-SA 3.0
2011-08-17T20:44:41.047Z
Perhaps this is relevant: ["Random multiplicative processes: An elementary tutorial" by S. Redner](http://denali.phys.uniroma1.it/twiki/pub/TNTgroup/AngeloVulpiani/redner.pdf)?  You might want to also check out ["From gene families and genera to incomes and internet file sizes: Why power laws are so common in nature" by W. Reed and B. Hughes](http://www.angelfire.com/nv/telka/transfer/powerlaw_expl.pdf).
user834
<pr.probability><st.statistics>
Multiplication of normal distributions
Mahdi Cheraghchi
# Multiplication of normal distributions

(Asked by: username_0 on 2011-08-13)

Suppose X\_1, ..., X\_k are iid standard Gaussian variables, for some k > 1.

Then, what is the distribution of X := X\_1 \* ... \* X\_k ? Can it be approximated by a Gaussian, maybe for large k ?

**Tags:** pr.probability, st.statistics
> **Comments:**
> **username_1**: I am failing to see how this questions is related to theoretical computer science. If this is only statistics/probability you can ask on http://stats.stackexchange.com/.  
> **username_1**: The FAQ of this site: http://cstheory.stackexchange.com/faq.  
> **username_2**: I agree that this is off-topic here. But this page might be useful (covers some special cases):  http://mathworld.wolfram.com/NormalProductDistribution.html  
> **username_0**: Polynomials of Gaussian random variables arise in theoretical CS often, especially in Fourier analysis of Boolean functions, inapproximability and so on. I'm asking here since I believe TCS researchers may have more "CS-friendly" answers to this question (e.g., involving approximations, quantitative bounds, etc).  
> **username_3**: I agree, MCH. I don't understand the objections.  
> **username_4**: Although MO or stats.SE might be a better place to ask this question, I don't think it is off-topic for cstheory, approximating distributions is also studied in TCS.  
> **username_2**: @MCH: Perhaps you could edit the question to emphasise TCS connections and TCS aspects, to make it more obvious that it is on-topic here? And now that we have already some answers – and it seems that the general case is very complicated – perhaps you could also tell us more about the specific requirements of your application?  
> **username_5**: Perhaps this is relevant: ["Random multiplicative processes: An elementary tutorial" by S. Redner](http://denali.phys.uniroma1.it/twiki/pub/TNTgroup/AngeloVulpiani/redner.pdf)?  You might want to also check out ["From gene families and genera to incomes and internet file sizes: Why power laws are so common in nature" by W. Reed and B. Hughes](http://www.angelfire.com/nv/telka/transfer/powerlaw_expl.pdf).  

---

## 2 answers

### Answer by username_6 on 2011-08-17. Score: 4 | Accepted answer

When the $X\_i$ are iid for some distribution with non-negative essential support (*i.e.*, the probability of a negative value is zero) and the distribution of $\log(X)$ has a finite variance, then the Central Limit Theorem easily implies that $X^\* = \log(X\_1 \times \cdots \times X\_k)$ approximately has a normal (Gaussian) distribution for sufficiently large $k$.

The problem here is that each $X\_i$ has a 50% chance of being negative, precluding (easy) analysis with logarithms. Nevertheless, only two things can happen (almost surely, after neglecting the zero chance that any of the $X\_i$ equals $0$): either an even number of the $X\_i$ are negative, in which case the product is positive, or an odd number of the $X\_i$ are negative, in which case the product is negative. Each case has a 50% chance of happening. Whence, the probability density of the product has two parts (symmetric to each other about 0) and each part looks like the product of draws from a [half-normal distribution](http://en.wikipedia.org/wiki/Half-normal_distribution). A calculation shows that the log of a half-normal distribution has finite variance, allowing us to apply the preceding observation and conclude that for sufficiently large $k$, the distribution of $\log(X^\*)$ *conditional* on $X^\* \gt 0$ is approximately Gaussian and the distribution of $\log(-X^\*)$ conditional on $X^\* \lt 0$ is exactly the same. ($X^\*$ is an equal mixture of variables $Y$ and $Z$ where $Y$ is approximately lognormal and $-Z$ has the same distribution as $Y$.)

In effect, the distribution of $X^\*$ has two pieces: the positive piece is approximately lognormal and the negative piece is its reflection about 0. This makes it bimodal; all odd moments are $0$; and small even moments can be approximated from the Central Limit Theorem. In particular, the mean of the positive part of $\log(X^\*)$ equals $-(\log(2) + \gamma)k/2$, approximately $-0.635181 k$, and its variance equals $\pi^2 k/8$, approximately $1.2337 k$.

Simulations indicate this approximation is good for $k \gt 100$, more or less. Considerable negative skewness in the distribution of $\log(|X^\*|)$ is evident for smaller $k$.

![Histogram for k=100](https://i.sstatic.net/3lF8i.png)

*This (unnormalized) histogram documents 100,000 independent realizations of $\log(|X^\*|)$ with $k=100$. Very slight negative skewness is still visible, but the general Gaussian shape is apparent, centered as expected around -63.5 with variance 123.3 (standard deviation 11.1).*

It is evident from the wide range of highly negative logarithms that an attempt to plot the distribution of $X^\*$ itself would be useless: it would show as a unit spike at 0. If we normalize it, though, a plot is informative. Specifically, consider the "signed geometric mean" $\text{Sign}(X^\*)\exp(\log(|X^\*|)/k)$:

![Histogram of the signed geometric mean](https://i.sstatic.net/2X6Rq.png)

*This histogram displays the same data as the previous one, but normalized as signed geometric means. The shape on the right is a lognormal distribution; the shape on the left is its reflection about 0.*


---

### Answer by username_7 on 2011-08-13. Score: 10

The distribution of the multiplication of independent Gaussians is not close to a Gaussian.

There are various ways to convince yourself in this. E.g., compute the expectation and the variance of the product from the expectation and the variance of the individual variables.
This gives you a candidate Gaussian if there were a Gaussian approximation.
Now, think of some simple properties of the product (e.g., the probability it is 0) and see if your candidate is good enough as an approximation.

> **Comments:**
> **username_0**: Maybe that's true, since the 4th and higher moments don't match and the discrepancy becomes greater as k grows. But is this distribution studied somewhere, say for k=2 ?  
> **username_7**: Can whoever down-voted my answer please provide an explanation for why they did so?  
> **username_7**: MCH, I trust that you saw that Jukka answered your question above.  
> **username_0**: Yes, indeed the link answers the question for k=2, 3. It's interesting to see that the functions involved get increasingly complex even for very small k. This seems inevitable if one wishes to find out the exact distribution. It would be nice though to have some quantitative estimates about the distribution, I'll dig up the literature to see if something's already known. Thanks!  
> **username_8**: Please explain before giving downvotes. I agree with Dana's comment, and am also puzzled why the answer would get a downvote.  
> **username_9**: @Suresh I am not the downvoter, but I did consider it. The reason I considered it is because I think the question if off-topic and from meta discussions like [this](http://meta.cstheory.stackexchange.com/q/601/1037) I got the impression that we should discourage answering off-topic questions.  
> **username_7**: My interpretation is that down-voting an answer is like flagging it false, misleading, irrelevant etc. If you don't want to encourage something don't up-vote it.  
> **username_10**: Or down-voting can mean “this should not have been posted.”  People interpret the same thing differently, and you cannot tell others when to down-vote answers and when not to.  
> **username_6**: I agree that this question belongs elsewhere.  First choice should be stats, because it's a natural one for statisticians (and it has been tagged as statistical), but math would be ok too.  
> **username_4**: @whuber: I understand that you are a high rep user and moderator on stats.SE and of course this might be more suitable for stats.SE, but it is also fine here, mathematical questions related to CS are in inside the scope of cstheory, this is not out of scope on cstheory. Often questions get cross-posted on cstheory and MO and that is fine.  
> **username_4**: @Artem, I don't find it reasonable to down vote an answer for answering a question which is not *clearly* off-topic, this does not look like a homework question to me, in any case, I think there was an agreement that it is not OK to down vote without commenting and explaining the reason. @ all, it would be more reasonable to continue on meta if there are disagreements and you want to continue discussing them.  
> **username_10**: @Kaveh: No matter what you say, some people down-vote without stating reasons or for reasons you do not find reasonable.  You have to accept the plain fact that some people act in a way which you find unreasonable.  Also, I had thought that the question was obviously off topic until I saw that a wrong answer was posted (which was deleted afterward).  Honestly speaking, I still think that the question is off topic, but I do not want to waste energy arguing that.  
> **username_4**: @Tsuyoshi, you are correct about down votes, I was just explaining the norms and agreements (no one argued against it in the Town Hall meeting AFAIR), of course people can violate them if they want, as we see in this case, and we can complain about those violations, which is what we are doing here.  
> **username_9**: @Kaveh I have started a meta discussion about [this already](http://meta.cstheory.stackexchange.com/questions/1264/conventions-on-downvoting). Note however, that I did not downvote this answer, I just considered doing so when I thought the question was off-topic. Let's discuss more on the linked meta question.  
> **username_6**: @Kaveh That's ok.  I just didn't recognize any explicit CS application in this question.  

---

