<p>One of the first things that many people get wrong with MIS of direct lighting is that you have to always consider the same light source for both light sampling and BSDF sampling. For example, if you sampled Light <span class="math-container">$L_i$</span> during light sampling and the ray spawned from BSDF sampling hit <span class="math-container">$L_j$</span>, you cannot mix them together since the probabilities cannot be used together for weights. </p>

CC BY-SA 4.0
2019-01-08T16:08:54.213Z
That answers my point mentioned in the edit portion. The main problem still remains though.
gallickgunner
CC BY-SA 4.0
2019-01-08T14:46:33.790Z
2019-01-08T14:46:33.790Z
sriravic
<p>Throughout my answer I'll sometimes refer to some results in <a href="https://sites.fas.harvard.edu/~cs278/papers/veach.pdf" rel="nofollow noreferrer">https://sites.fas.harvard.edu/~cs278/papers/veach.pdf</a> by using [MIS,section_number].</p>

<p>You can skip the following derivation if you don't care about the mathematical explanation of why using MIS to combine estimators is valid.
I'll have to start with what the purpose of MIS is. The general idea is that you want to estimate some integral <span class="math-container">$I=\int_{\Omega}{f(x)\,d\mu(x)}$</span> through Monte Carlo, and you have various sampling pdfs (in our case a pdf for sampling points on the light, and a pdf for sampling directions from the brdf). Additionally you want to sample using more than one technique and then combine the contributions optimally (in some sense). For simplicity I will restrict myself to the case where you have only two pdfs: <span class="math-container">$p_L(x), p_B(x)$</span> with respect to the same measure <span class="math-container">$\mu(x)$</span>.</p>

<p>Assume that you also have the weighting functions <span class="math-container">$w_1(x), w_2(x) : \Omega \rightarrow [0,1], w_1(x) + w_2(x) = 1$</span> and also <span class="math-container">$n$</span> independently and identically distributed (i.i.d.) samples <span class="math-container">$x_{p_L,i}: i=1,...,n$</span> according to <span class="math-container">$p_L$</span>, and <span class="math-container">$m$</span> i.i.d. samples <span class="math-container">$x_{p_B,j} : j=1,...,m$</span> according to <span class="math-container">$p_B$</span>, we can use Monte Carlo with MIS to estimate the integral <span class="math-container">$I$</span>:
<span class="math-container">$$ I = \int_{\Omega}{f(x)\,d\mu(x)} = \int_{\Omega}{f(x)(w_1(x)+w_2(x))\,d\mu(x)}  \\
= \int_{\Omega}{\frac{w_1(x)f(x)}{p_L(x)}p_L(x)\,d\mu(x)} + \int_{\Omega}{\frac{w_2(x)f(x)}{p_B(x)}p_B(x)\,d\mu(x)} \\
= E\left[\frac{w_1(X_{p_L})f(X_{p_L})}{p_L(X_{p_L})}\right] + E\left[\frac{w_2(X_{p_B})f(X_{p_B})}
{p_B(X_{p_B})}\right] \\
= \frac{1}{n}\sum_{i=1}^{n}{E\left[\frac{w_1(X_{p_L,i})f(X_{p_L,i})}{p_L(X_{p_L,i})}\right]} + \frac{1}{m}\sum_{j=1}^{m}{E\left[\frac{w_2(X_{p_B,j})f(X_{p_B,j})}
{p_B(X_{p_B,j})}\right]} \\
= E\left[\frac{1}{n}\sum_{i=1}^{n}{\frac{w_1(X_{p_L,i})f(X_{p_L,i})}{p_L(X_{p_L,i})}}\right] + E\left[\frac{1}{m}\sum_{j=1}^{m}{\frac{w_2(X_{p_B,j})f(X_{p_B,j})}
{p_B(X_{p_B,j})}}\right] \\
\approx \frac{1}{n}\sum_{i=1}^{n}{\frac{w_1(x_{p_L,i})f(x_{p_L,i})}{p_L(x_{p_L,i})}} 
+ \frac{1}{m}\sum_{j=1}^{m}{\frac{w_2(x_{p_B,j})f(x_{p_B,j})}
{p_B(x_{p_B,j})}}
$$</span>
Where the second equality holds because <span class="math-container">$w_1(x)+w_2(x)=1$</span>, the fourth follows from the definition of expected value, the fifth holds since <span class="math-container">$X_{p_L,i}$</span> are independently and identically distributed (i.i.d.) according to <span class="math-container">$p_L$</span> and similarly <span class="math-container">$X_{p_B,i}$</span> are i.i.d. according to <span class="math-container">$p_B$</span>. The sixth holds because of the properties of the expectation (which follows from the fact that integration is a linear operator). The last approximate equality follows from the strong law of large numbers (the average of the samples converges almost surely to the expected value with the number of samples going to infinity).</p>

<p>Having this result the first thing to note is that both pdfs are defined with respect to the same measure <span class="math-container">$\mu(x)$</span>. However I believe that in your code and overview you pdf for sampling the light <span class="math-container">$p_L$</span> is defined with respect to the area measure, whereas the pdf for sampling the brdf <span class="math-container">$p_B$</span> is defined with respect to the solid angle measure. The relationship between those being <span class="math-container">$d\sigma(x\rightarrow y) = \frac{\cos\theta_y}{||x-y||^2}dA(y)$</span>, where <span class="math-container">$x \in \mathbb{R}^3$</span> is the current point for which you are computing the illumination (that is your intersection point), and <span class="math-container">$y \in \mathbb{R}^3$</span> is a point on the surface of some light source. Additionally <span class="math-container">$(x\rightarrow y) = \frac{y-x}{||y-x||}$</span> is simply the unit direction vector from <span class="math-container">$x$</span> to <span class="math-container">$y$</span>, <span class="math-container">$\cos\theta_y = n_y \cdot (y\rightarrow x)$</span> is the cosine of the angle between the normal <span class="math-container">$n_y$</span> (also unit length) of the light surface at <span class="math-container">$y$</span> and <span class="math-container">$-(x\rightarrow y)$</span>. More specifically what you have produced as light pdf is really the probability of picking the light <span class="math-container">$weights[i]/sum$</span> multiplied by the probability of picking uniformly a point on the light source that you have chosen <span class="math-container">$1/area$</span>. The solid angle to area (or vice versa) conversion seems to be missing in your code and your overview, that is your pdf is with respect to the area measure, and you combine it with a pdf (the brdf's) that is with respect to the solid angle measure, which is clearly wrong as you can see from my derivation above. To get the  brdf pdf with respect to the area measure and not the solid angle measure you can use: <span class="math-container">$p_{\omega}(\omega)\,d\sigma(\omega) = p_A(y)\,dA(y)$</span> then using the relationship between the measures <span class="math-container">$p_A(y) = p_{\omega}(\omega)\,d\sigma(\omega) / dA(y) = p_{\omega}(\omega) \frac{\cos\theta_y}{||x-y||^2}$</span>. You can refer to [MIS, 2.3] equation (9). For a formal derivation of the relationship between the two measures you can refer to <a href="http://www.dgp.toronto.edu/~lessig/dissertation/files/area_formulation.pdf" rel="nofollow noreferrer">http://www.dgp.toronto.edu/~lessig/dissertation/files/area_formulation.pdf</a> , another possible derivation of the same fact can be done through the divergence theorem.</p>

<p>Now as far as the practical part goes - you need to know when to apply the mis weighting. There are a few cases which occur. Initially for the very first ray, you should not use MIS since you shouldn't sample any light directly. Additionally if the last bounce was fully deterministic (ideal reflection/refraction) then you also should not use MIS, since sampling the light is useless in that case. Finally if you hit a light through sampling the bsdf, you should use MIS to add this contribution, and convert the bsdf pdf wrt the area measure, and then combine it with the probability of sampling this point on this light by using the light pdf and use the area formulation estimator of the rendering equation. When you sample a light, you should also use MIS, once again computing the area formulation estimator. Finally if you have point lights, those can be sampled only through NEE, so you should not use MIS.</p>

<p>Note that there may be more mistakes that you have. Also I didn't understand anything of your explanation past "After that I come to BRDF sampling.".</p>

CC BY-SA 4.0
2019-02-24T18:33:40.167Z
Thanks for the detailed write-up. So in short (1) I need to compute both light and brdf samples using integral over area not solid angle. And (2) I also need to compute the BRDF PDF w.r.t to the area. I was actually doing (1) but changed it to integral over solid angle and noticed images getting brighter. Well anyways, will report back shortly.
gallickgunner
CC BY-SA 4.0
2019-02-24T18:47:44.437Z
You can decide on one of the two formulations: either area or solid angle. That means that all of your pdfs (even outside of the weights) should match this formulation, as well the rendering equation formulation that you use and consequently the estimator that you will compute. The good thing is that nothing except for the pdfs in your weights needs to change. Since I think that most of your stuff is already in solid angle formulation (your brdf pdf and the estimators), then just convert the light pdf wrt solid angle (mult by $r^2/\cos\theta_y$).
lightxbulb
CC BY-SA 4.0
2019-02-24T19:43:53.237Z
Hey thanks for the answer. I think most of the problems have been resolved. Check the update images. The fireflies seem no less though or it's just that there are too many of them to make a difference. I'll try with different scene parameters. I also don't get what you said about the first ray should not use MIS. I think you meant for naive path tracing. I'm using NEE.
gallickgunner
CC BY-SA 4.0
2019-02-24T20:02:04.853Z
The first ray (emanating from the camera) should not use NEE, since you will hit anything visible either way. The fact that the fireflies do not even diminish is strange (it's not impossible though), so I still have my doubts.
lightxbulb
CC BY-SA 4.0
2019-02-24T21:07:54.687Z
About fireflies diminishing, I think the real reason for fireflies in my scene is because the reflective sphere shows an image of bright light source. This is very small as compared to the actual size of the light sources. Hence when the rays hitting the walls bounce, only a very few of them end up striking the light source image on the reflective sphere giving an incredibly high color in some samples. This just sounds like I am double dipping lights (1 from NEE, 2nd from reflective sphere) Sounds wrong but never seen such a case where people ignore the reflection of light in GI.
gallickgunner
CC BY-SA 4.0
2019-02-24T21:11:24.977Z
I don't really understand how MIS is gonna help me reduce fireflies in this case? Since it's acutally supposed to reduce variance when shooting at small light sources and for direct illumination if I got the gist correct.
gallickgunner
CC BY-SA 4.0
2019-02-24T21:59:57.103Z
MIS is not only there to help you reduce variance from sampling small light sources through brdf sampling, it works the other way around too - giving more weight when sampling a large light through a tight brdf as opposed to sampling the light by direct light sampling (which can be outside the cone of the brdf). Note that MIS can in fact also increase variance (if both estimators are bad at some point). I can't tell why your scene has so many fireflies, since the illumination seems smooth for the most part. I am assuming a phong with a power that's too high, or something's wrong.
lightxbulb
CC BY-SA 4.0
2019-02-24T22:05:45.687Z
That's actually true. The reflective sphere isn't modeled as a specular surface but a glossy one with a high phong exponent, About the MIS part, I still think it wouldn't help much here since all the surfaces are diffuse except for the reflective one. And diffuse surfaces have a very generic brdf, So there won't be any noticable difference. The thing you are talking about (tight brdfs) would be in case of glossy and near mirror surfaces. But the fireflies here are appearing on diffuse surfaces.
gallickgunner
CC BY-SA 4.0
2019-02-24T22:21:48.073Z
The fireflies are appearing on diffuse surfaces due to the reflection off the high exponent sphere. As you can see it is riddled with fireflies, and I am unsure whether this is just due to a mistake in the implementation.
lightxbulb
CC BY-SA 4.0
2019-02-24T22:32:35.680Z
Yea that's what I said earlier. Anyways I'll try to debug it again and see If I did any sort of mistakes. Thanks for the help on MIS tho.
gallickgunner
CC BY-SA 4.0
2019-02-24T17:59:53.187Z
2019-02-24T17:59:53.187Z
lightxbulb
<p>So I recently implemented Multiple Importance Sampling in my path tracer which was based on next event estimation.</p>

<p>The problem is without MIS I get images like,</p>

<p><a href="https://i.sstatic.net/7SL0B.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/7SL0B.png" alt="enter image description here"></a></p>

<p>This is obtained by setting<br>
<code>light_sample *=  1/light_pdf;  // Note no mis_weight</code><br>
and just returning this sample alone. No BRDF sampling.</p>

<p>Where as with MIS as defined above I get darker images like,
<a href="https://i.sstatic.net/QYyJw.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QYyJw.png" alt=""></a></p>

<p>The reason is surely the <code>mis_weight</code> factor. From what I gathered over the internet my MIS code is ok but theoretically it doesn't seem right. For example suppose we performed Light Sampling first and obtained a <code>mis_weight</code>. After that when we tried BRDF sampling, the ray didn't intersect any light source resulting in <code>light_pdf = 0</code>. We neglected this estimate. Since we neglected this estimate isn't it wrong to weight our light sample by <code>mis_weight</code> when we didn't even use multiple estimators, plus how is the sum of weights equal to 1 when we didnt even estimate anything using the BRDF pdf?</p>

<p>Imo, <code>mis_weight</code> factor in <code>light_sample</code> should only be used when BRDF sampling also results in a light ray that intersects the light source.</p>

<p>Can anybody explain if these results are ok or there is something wrong with the code?</p>

<p><strike><strong>EDIT</strong>:- There is another case which is a little confusing. I am currently using a heuristic (basically power/distance) to choose 1 light out of multiple lights. What if Light sampling and BRDF sampling choose different light sources. Is MIS still valid then? Since the PDF would change resulting the weights not being able to sum to 1 anymore</strike></p>

<p><strong>EDIT2</strong>:- So Stefan pointed out a mistake <a href="https://computergraphics.stackexchange.com/questions/8469/naive-path-tracer-produces-much-darker-images-than-explicit-light-sampling?noredirect=1&amp;lq=1">here</a>, that I was clamping radiance. And this solved that issue and changed the results I'm getting for MIS. I have double checked PBRT's implementation and it is similar to mine. I'm still getting darker images using MIS <strike>however now I'm getting more fireflies and I think I read that MIS reduces fireflies?</strike> Updated the images. <strong>It seems fireflies are less in MIS however, the reflection seems harder to converge.</strong> </p>

<p>Note the reflection is harder to converge. Both images are 1000 spp. Top is without MIS.</p>

<p><a href="https://i.sstatic.net/WeiZN.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/WeiZN.png" alt="enter image description here"></a>
<a href="https://i.sstatic.net/PPnyh.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/PPnyh.png" alt="enter image description here"></a></p>

<p>I'm removing the snippet and adding a link to github for the whole kernel. The core functions are <code>evaluateDirectLighting</code>, <code>shading</code>, <code>sampleLights</code>. Link to code is "<a href="https://github.com/gallickgunner/Yune/blob/master/progressive-PT.cl" rel="nofollow noreferrer">here</a>"</p>

<p>--<strike>Found a mistake. I was using the direct lighting equation when calculating the brdf sample. (multiplying by <span class="math-container">$\cos(\theta^{\prime})$</span> and dividing by <span class="math-container">$r^2$</span>). Removed it since we initially sampled through BSDF which is integral over solid angle not area. The image got a little brighter. Still don't know if MIS is working as intended.</strike> See answer.</p>

<p><strong>EDIT3</strong>:- Added a release as well. So if anybody wants, they can try changing the code in "*.cl" file and run the program to see the results. (You must have an OpenCL 1.1 supported GPU or CPU)</p>

<p><strong>EDIT4</strong>:- Here's an overview of what I'm doing now.</p>

<p>First I choose a single light source out of multiple using simple heuristic scheme like the distance, intensity, area and the cosine falloff angles. I appropriately set the weights for the light like this</p>

<p><span class="math-container">$light\_pdf = weight/area$</span></p>

<p>where weight is in range 0-1.</p>

<p>Next I trace the ray to see if light source is visible. If it is I calculate the <span class="math-container">$light\_sample$</span> using the direct lighting equation (integral over Area).</p>

<p>Then I calculate the BRDF PDF for this given ray. However I use Lafortune's algorithm for it. If a random number falls under the specular color I sample through the modfied Phong PDF else through Cosine.</p>

<p>The weights are computed using power heuristic,
<span class="math-container">$ weight = light\_pdf^2/(light\_pdf^2+brdf\_pdf^2)$</span></p>

<p>The MIS estimator is then calculated as</p>

<p><span class="math-container">$light\_sample = light\_sample * weight/light\_pdf$</span></p>

<p>After that I come to BRDF sampling. I again sample through either Phong/Cosine based on what I did earlier during light sample calculation. If the sampled ray doesn't hit any light source or not the <strong>same</strong> one. I set the <span class="math-container">$brdf\_sample$</span> to zero. If it hits, I set the <span class="math-container">$light\_pdf$</span> to same value as before. Calculate the weights like mentioned above and calculate brdf sample using the original equation (integral over solid angle).</p>

<p><strong>EDIT5</strong>:- After lightxbulb suggestions, I think the problem has resolved.</p>

<p>Note the images might look a whole lot different but that's cause I implemented Tonemapping + Gamma Correction in the meantime :)
With MIS 700spp
<a href="https://i.sstatic.net/1ZWKG.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1ZWKG.jpg" alt="enter image description here"></a></p>

<p>Without MIS 700spp
<a href="https://i.sstatic.net/jWrUa.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jWrUa.jpg" alt="enter image description here"></a></p>

CC BY-SA 4.0
2019-01-07T06:26:54.650Z
You shouldn't include the cosine foreshortening term in your `brdf_pdf`.
Hubble
CC BY-SA 4.0
2019-01-07T09:57:05.943Z
Im using cosine weighted hemispherical sampling so the BRDF PDF is supposed to be `cos(theta)/pi`
gallickgunner
CC BY-SA 4.0
2019-01-11T07:38:52.690Z
May be a coincidence but are your diffuse surfaces exactly PI times darker ?
PaulHK
CC BY-SA 4.0
2019-01-11T10:33:19.567Z
@PaulHK - What would that imply? I don't think it's actually PI times darker, its just `mis_weight` times darker/brighter. Because as I said as soon as i remove mis_wieght and just divide by `light_pdf` I get the brighter image.
gallickgunner
CC BY-SA 4.0
2019-01-14T02:07:38.130Z
A shot in the dark really, I have seen other question on here were the solution was surprisingly that.
PaulHK
CC BY-SA 4.0
2019-01-14T22:38:21.970Z
What do you get if you take out your light source choosing heuristic?
trichoplax is on Codidact now
CC BY-SA 4.0
2019-01-15T09:34:48.350Z
@trichoplax - If i removed it, this gets a little messy. This is how I do it. I'm now accumulating color from two light sources but it's a single sample ( we don't want to divide by 2). However for every light source I also take a `brdf_sample`. This accounts as 2 BRDF samples. Hence I use MIS with the formula accounting for 1  light sample and 2 BRDF samples. Result seem almost same to me. Note if i assume those 2 brdf samples as 1, results are very bright, like way bright so I think the first approach is right. Updated github so you can see how I did it.
gallickgunner
CC BY-SA 4.0
2019-02-20T09:46:20.497Z
I need to update this. Should I edit out the previous stuff or leave it as it is so people can follow the changes?
gallickgunner
CC BY-SA 4.0
2019-02-24T14:17:55.280Z
Could you add a small LaTeX derivation as to how you compute your weights, and at what events you do so, and the the final estimator you use. Basically a small explanation as to when and how you do direct light sampling, and brdf sampling and then the way you combine those. I can also link you some code of mine with MIS in glsl if you want.
lightxbulb
CC BY-SA 4.0
2019-02-24T14:45:49.233Z
@lightxbulb - I actually posted the link to the whole code incase you overlooked it.  Check from line 471. Didn't want to add here since it's gotten messy and long as it is. The weights are computed using power heuristic (1 line function at the end of the file). If you still want I can try posting here as well.
gallickgunner
CC BY-SA 4.0
2019-02-24T15:08:49.153Z
@gallickgunner I saw that you linked the code, however, I'd rather see the theoretical explanation of what you're doing rather than go through the code. Just a high level overview is ok.
lightxbulb
CC BY-SA 4.0
2019-02-24T15:25:08.277Z
Note that I have gone through your code, and I believe there are a number of mistakes. I think that if you do a small overview of the relevant parts then: 1) I'll be certain it's not me misunderstanding what your code does, 2) I'll be able to explain why theoretically what you are doing is incorrect (without talking about the implementation specifically). I just think you've misunderstood some parts of the derivation of MIS. If we can clarify which parts, then I can do a detailed writeup.
lightxbulb
CC BY-SA 4.0
2019-02-24T15:49:35.353Z
@lightxbulb - Check now.
gallickgunner
<rendering><pathtracing><monte-carlo><importance-sampling>
Multiple Importance Sampling in Path tracer produces Dark Images
gallickgunner
# Multiple Importance Sampling in Path tracer produces Dark Images

(Asked by: username_0 on 2019-01-04)

So I recently implemented Multiple Importance Sampling in my path tracer which was based on next event estimation.

The problem is without MIS I get images like,

[![enter image description here](https://i.sstatic.net/7SL0B.png)](https://i.sstatic.net/7SL0B.png)

This is obtained by setting  
`light_sample *= 1/light_pdf; // Note no mis_weight`  
and just returning this sample alone. No BRDF sampling.

Where as with MIS as defined above I get darker images like,
[![](https://i.sstatic.net/QYyJw.png)](https://i.sstatic.net/QYyJw.png)

The reason is surely the `mis_weight` factor. From what I gathered over the internet my MIS code is ok but theoretically it doesn't seem right. For example suppose we performed Light Sampling first and obtained a `mis_weight`. After that when we tried BRDF sampling, the ray didn't intersect any light source resulting in `light_pdf = 0`. We neglected this estimate. Since we neglected this estimate isn't it wrong to weight our light sample by `mis_weight` when we didn't even use multiple estimators, plus how is the sum of weights equal to 1 when we didnt even estimate anything using the BRDF pdf?

Imo, `mis_weight` factor in `light_sample` should only be used when BRDF sampling also results in a light ray that intersects the light source.

Can anybody explain if these results are ok or there is something wrong with the code?

**EDIT**:- There is another case which is a little confusing. I am currently using a heuristic (basically power/distance) to choose 1 light out of multiple lights. What if Light sampling and BRDF sampling choose different light sources. Is MIS still valid then? Since the PDF would change resulting the weights not being able to sum to 1 anymore

**EDIT2**:- So Stefan pointed out a mistake [here](https://computergraphics.stackexchange.com/questions/8469/naive-path-tracer-produces-much-darker-images-than-explicit-light-sampling?noredirect=1&lq=1), that I was clamping radiance. And this solved that issue and changed the results I'm getting for MIS. I have double checked PBRT's implementation and it is similar to mine. I'm still getting darker images using MIS however now I'm getting more fireflies and I think I read that MIS reduces fireflies? Updated the images. **It seems fireflies are less in MIS however, the reflection seems harder to converge.**

Note the reflection is harder to converge. Both images are 1000 spp. Top is without MIS.

[![enter image description here](https://i.sstatic.net/WeiZN.png)](https://i.sstatic.net/WeiZN.png)
[![enter image description here](https://i.sstatic.net/PPnyh.png)](https://i.sstatic.net/PPnyh.png)

I'm removing the snippet and adding a link to github for the whole kernel. The core functions are `evaluateDirectLighting`, `shading`, `sampleLights`. Link to code is "[here](https://github.com/username_0/Yune/blob/master/progressive-PT.cl)"

--Found a mistake. I was using the direct lighting equation when calculating the brdf sample. (multiplying by $\cos(\theta^{\prime})$ and dividing by $r^2$). Removed it since we initially sampled through BSDF which is integral over solid angle not area. The image got a little brighter. Still don't know if MIS is working as intended. See answer.

**EDIT3**:- Added a release as well. So if anybody wants, they can try changing the code in "\*.cl" file and run the program to see the results. (You must have an OpenCL 1.1 supported GPU or CPU)

**EDIT4**:- Here's an overview of what I'm doing now.

First I choose a single light source out of multiple using simple heuristic scheme like the distance, intensity, area and the cosine falloff angles. I appropriately set the weights for the light like this

$light\\_pdf = weight/area$

where weight is in range 0-1.

Next I trace the ray to see if light source is visible. If it is I calculate the $light\\_sample$ using the direct lighting equation (integral over Area).

Then I calculate the BRDF PDF for this given ray. However I use Lafortune's algorithm for it. If a random number falls under the specular color I sample through the modfied Phong PDF else through Cosine.

The weights are computed using power heuristic,
$ weight = light\\_pdf^2/(light\\_pdf^2+brdf\\_pdf^2)$

The MIS estimator is then calculated as

$light\\_sample = light\\_sample \* weight/light\\_pdf$

After that I come to BRDF sampling. I again sample through either Phong/Cosine based on what I did earlier during light sample calculation. If the sampled ray doesn't hit any light source or not the **same** one. I set the $brdf\\_sample$ to zero. If it hits, I set the $light\\_pdf$ to same value as before. Calculate the weights like mentioned above and calculate brdf sample using the original equation (integral over solid angle).

**EDIT5**:- After username_4 suggestions, I think the problem has resolved.

Note the images might look a whole lot different but that's cause I implemented Tonemapping + Gamma Correction in the meantime :)
With MIS 700spp
[![enter image description here](https://i.sstatic.net/1ZWKG.jpg)](https://i.sstatic.net/1ZWKG.jpg)

Without MIS 700spp
[![enter image description here](https://i.sstatic.net/jWrUa.jpg)](https://i.sstatic.net/jWrUa.jpg)

**Tags:** rendering, pathtracing, monte-carlo, importance-sampling

> **Comments:**
> **username_1**: You shouldn't include the cosine foreshortening term in your `brdf_pdf`.  
> **username_0**: Im using cosine weighted hemispherical sampling so the BRDF PDF is supposed to be `cos(theta)/pi`  
> **username_2**: May be a coincidence but are your diffuse surfaces exactly PI times darker ?  
> **username_0**: @username_2 - What would that imply? I don't think it's actually PI times darker, its just `mis_weight` times darker/brighter. Because as I said as soon as i remove mis_wieght and just divide by `light_pdf` I get the brighter image.  
> **username_2**: A shot in the dark really, I have seen other question on here were the solution was surprisingly that.  
> **username_3**: What do you get if you take out your light source choosing heuristic?  
> **username_0**: @username_3 - If i removed it, this gets a little messy. This is how I do it. I'm now accumulating color from two light sources but it's a single sample ( we don't want to divide by 2). However for every light source I also take a `brdf_sample`. This accounts as 2 BRDF samples. Hence I use MIS with the formula accounting for 1  light sample and 2 BRDF samples. Result seem almost same to me. Note if i assume those 2 brdf samples as 1, results are very bright, like way bright so I think the first approach is right. Updated github so you can see how I did it.  
> **username_0**: I need to update this. Should I edit out the previous stuff or leave it as it is so people can follow the changes?  
> **username_4**: Could you add a small LaTeX derivation as to how you compute your weights, and at what events you do so, and the the final estimator you use. Basically a small explanation as to when and how you do direct light sampling, and brdf sampling and then the way you combine those. I can also link you some code of mine with MIS in glsl if you want.  
> **username_0**: @username_4 - I actually posted the link to the whole code incase you overlooked it.  Check from line 471. Didn't want to add here since it's gotten messy and long as it is. The weights are computed using power heuristic (1 line function at the end of the file). If you still want I can try posting here as well.  
> **username_4**: @username_0 I saw that you linked the code, however, I'd rather see the theoretical explanation of what you're doing rather than go through the code. Just a high level overview is ok.  
> **username_4**: Note that I have gone through your code, and I believe there are a number of mistakes. I think that if you do a small overview of the relevant parts then: 1) I'll be certain it's not me misunderstanding what your code does, 2) I'll be able to explain why theoretically what you are doing is incorrect (without talking about the implementation specifically). I just think you've misunderstood some parts of the derivation of MIS. If we can clarify which parts, then I can do a detailed writeup.  
> **username_0**: @username_4 - Check now.  

---

## 2 answers

### Answer by username_4 on 2019-02-24. Score: 4 | Accepted answer

Throughout my answer I'll sometimes refer to some results in <https://sites.fas.harvard.edu/~cs278/papers/veach.pdf> by using [MIS,section\_number].

You can skip the following derivation if you don't care about the mathematical explanation of why using MIS to combine estimators is valid.
I'll have to start with what the purpose of MIS is. The general idea is that you want to estimate some integral $I=\int\_{\Omega}{f(x)\,d\mu(x)}$ through Monte Carlo, and you have various sampling pdfs (in our case a pdf for sampling points on the light, and a pdf for sampling directions from the brdf). Additionally you want to sample using more than one technique and then combine the contributions optimally (in some sense). For simplicity I will restrict myself to the case where you have only two pdfs: $p\_L(x), p\_B(x)$ with respect to the same measure $\mu(x)$.

Assume that you also have the weighting functions $w\_1(x), w\_2(x) : \Omega \rightarrow [0,1], w\_1(x) + w\_2(x) = 1$ and also $n$ independently and identically distributed (i.i.d.) samples $x\_{p\_L,i}: i=1,...,n$ according to $p\_L$, and $m$ i.i.d. samples $x\_{p\_B,j} : j=1,...,m$ according to $p\_B$, we can use Monte Carlo with MIS to estimate the integral $I$:
$$ I = \int\_{\Omega}{f(x)\,d\mu(x)} = \int\_{\Omega}{f(x)(w\_1(x)+w\_2(x))\,d\mu(x)} \\
= \int\_{\Omega}{\frac{w\_1(x)f(x)}{p\_L(x)}p\_L(x)\,d\mu(x)} + \int\_{\Omega}{\frac{w\_2(x)f(x)}{p\_B(x)}p\_B(x)\,d\mu(x)} \\
= E\left[\frac{w\_1(X\_{p\_L})f(X\_{p\_L})}{p\_L(X\_{p\_L})}\right] + E\left[\frac{w\_2(X\_{p\_B})f(X\_{p\_B})}
{p\_B(X\_{p\_B})}\right] \\
= \frac{1}{n}\sum\_{i=1}^{n}{E\left[\frac{w\_1(X\_{p\_L,i})f(X\_{p\_L,i})}{p\_L(X\_{p\_L,i})}\right]} + \frac{1}{m}\sum\_{j=1}^{m}{E\left[\frac{w\_2(X\_{p\_B,j})f(X\_{p\_B,j})}
{p\_B(X\_{p\_B,j})}\right]} \\
= E\left[\frac{1}{n}\sum\_{i=1}^{n}{\frac{w\_1(X\_{p\_L,i})f(X\_{p\_L,i})}{p\_L(X\_{p\_L,i})}}\right] + E\left[\frac{1}{m}\sum\_{j=1}^{m}{\frac{w\_2(X\_{p\_B,j})f(X\_{p\_B,j})}
{p\_B(X\_{p\_B,j})}}\right] \\
\approx \frac{1}{n}\sum\_{i=1}^{n}{\frac{w\_1(x\_{p\_L,i})f(x\_{p\_L,i})}{p\_L(x\_{p\_L,i})}}
+ \frac{1}{m}\sum\_{j=1}^{m}{\frac{w\_2(x\_{p\_B,j})f(x\_{p\_B,j})}
{p\_B(x\_{p\_B,j})}}
$$
Where the second equality holds because $w\_1(x)+w\_2(x)=1$, the fourth follows from the definition of expected value, the fifth holds since $X\_{p\_L,i}$ are independently and identically distributed (i.i.d.) according to $p\_L$ and similarly $X\_{p\_B,i}$ are i.i.d. according to $p\_B$. The sixth holds because of the properties of the expectation (which follows from the fact that integration is a linear operator). The last approximate equality follows from the strong law of large numbers (the average of the samples converges almost surely to the expected value with the number of samples going to infinity).

Having this result the first thing to note is that both pdfs are defined with respect to the same measure $\mu(x)$. However I believe that in your code and overview you pdf for sampling the light $p\_L$ is defined with respect to the area measure, whereas the pdf for sampling the brdf $p\_B$ is defined with respect to the solid angle measure. The relationship between those being $d\sigma(x\rightarrow y) = \frac{\cos\theta\_y}{||x-y||^2}dA(y)$, where $x \in \mathbb{R}^3$ is the current point for which you are computing the illumination (that is your intersection point), and $y \in \mathbb{R}^3$ is a point on the surface of some light source. Additionally $(x\rightarrow y) = \frac{y-x}{||y-x||}$ is simply the unit direction vector from $x$ to $y$, $\cos\theta\_y = n\_y \cdot (y\rightarrow x)$ is the cosine of the angle between the normal $n\_y$ (also unit length) of the light surface at $y$ and $-(x\rightarrow y)$. More specifically what you have produced as light pdf is really the probability of picking the light $weights[i]/sum$ multiplied by the probability of picking uniformly a point on the light source that you have chosen $1/area$. The solid angle to area (or vice versa) conversion seems to be missing in your code and your overview, that is your pdf is with respect to the area measure, and you combine it with a pdf (the brdf's) that is with respect to the solid angle measure, which is clearly wrong as you can see from my derivation above. To get the brdf pdf with respect to the area measure and not the solid angle measure you can use: $p\_{\omega}(\omega)\,d\sigma(\omega) = p\_A(y)\,dA(y)$ then using the relationship between the measures $p\_A(y) = p\_{\omega}(\omega)\,d\sigma(\omega) / dA(y) = p\_{\omega}(\omega) \frac{\cos\theta\_y}{||x-y||^2}$. You can refer to [MIS, 2.3] equation (9). For a formal derivation of the relationship between the two measures you can refer to <http://www.dgp.toronto.edu/~lessig/dissertation/files/area_formulation.pdf> , another possible derivation of the same fact can be done through the divergence theorem.

Now as far as the practical part goes - you need to know when to apply the mis weighting. There are a few cases which occur. Initially for the very first ray, you should not use MIS since you shouldn't sample any light directly. Additionally if the last bounce was fully deterministic (ideal reflection/refraction) then you also should not use MIS, since sampling the light is useless in that case. Finally if you hit a light through sampling the bsdf, you should use MIS to add this contribution, and convert the bsdf pdf wrt the area measure, and then combine it with the probability of sampling this point on this light by using the light pdf and use the area formulation estimator of the rendering equation. When you sample a light, you should also use MIS, once again computing the area formulation estimator. Finally if you have point lights, those can be sampled only through NEE, so you should not use MIS.

Note that there may be more mistakes that you have. Also I didn't understand anything of your explanation past "After that I come to BRDF sampling.".

> **Comments:**
> **username_0**: Thanks for the detailed write-up. So in short (1) I need to compute both light and brdf samples using integral over area not solid angle. And (2) I also need to compute the BRDF PDF w.r.t to the area. I was actually doing (1) but changed it to integral over solid angle and noticed images getting brighter. Well anyways, will report back shortly.  
> **username_4**: You can decide on one of the two formulations: either area or solid angle. That means that all of your pdfs (even outside of the weights) should match this formulation, as well the rendering equation formulation that you use and consequently the estimator that you will compute. The good thing is that nothing except for the pdfs in your weights needs to change. Since I think that most of your stuff is already in solid angle formulation (your brdf pdf and the estimators), then just convert the light pdf wrt solid angle (mult by $r^2/\cos\theta_y$).  
> **username_0**: Hey thanks for the answer. I think most of the problems have been resolved. Check the update images. The fireflies seem no less though or it's just that there are too many of them to make a difference. I'll try with different scene parameters. I also don't get what you said about the first ray should not use MIS. I think you meant for naive path tracing. I'm using NEE.  
> **username_4**: The first ray (emanating from the camera) should not use NEE, since you will hit anything visible either way. The fact that the fireflies do not even diminish is strange (it's not impossible though), so I still have my doubts.  
> **username_0**: About fireflies diminishing, I think the real reason for fireflies in my scene is because the reflective sphere shows an image of bright light source. This is very small as compared to the actual size of the light sources. Hence when the rays hitting the walls bounce, only a very few of them end up striking the light source image on the reflective sphere giving an incredibly high color in some samples. This just sounds like I am double dipping lights (1 from NEE, 2nd from reflective sphere) Sounds wrong but never seen such a case where people ignore the reflection of light in GI.  
> **username_0**: I don't really understand how MIS is gonna help me reduce fireflies in this case? Since it's acutally supposed to reduce variance when shooting at small light sources and for direct illumination if I got the gist correct.  
> **username_4**: MIS is not only there to help you reduce variance from sampling small light sources through brdf sampling, it works the other way around too - giving more weight when sampling a large light through a tight brdf as opposed to sampling the light by direct light sampling (which can be outside the cone of the brdf). Note that MIS can in fact also increase variance (if both estimators are bad at some point). I can't tell why your scene has so many fireflies, since the illumination seems smooth for the most part. I am assuming a phong with a power that's too high, or something's wrong.  
> **username_0**: That's actually true. The reflective sphere isn't modeled as a specular surface but a glossy one with a high phong exponent, About the MIS part, I still think it wouldn't help much here since all the surfaces are diffuse except for the reflective one. And diffuse surfaces have a very generic brdf, So there won't be any noticable difference. The thing you are talking about (tight brdfs) would be in case of glossy and near mirror surfaces. But the fireflies here are appearing on diffuse surfaces.  
> **username_4**: The fireflies are appearing on diffuse surfaces due to the reflection off the high exponent sphere. As you can see it is riddled with fireflies, and I am unsure whether this is just due to a mistake in the implementation.  
> **username_0**: Yea that's what I said earlier. Anyways I'll try to debug it again and see If I did any sort of mistakes. Thanks for the help on MIS tho.  

---

### Answer by username_5 on 2019-01-08. Score: 2

One of the first things that many people get wrong with MIS of direct lighting is that you have to always consider the same light source for both light sampling and BSDF sampling. For example, if you sampled Light $L\_i$ during light sampling and the ray spawned from BSDF sampling hit $L\_j$, you cannot mix them together since the probabilities cannot be used together for weights.

> **Comments:**
> **username_0**: That answers my point mentioned in the edit portion. The main problem still remains though.  

---

