<p>There are, and I am looking forward to seeing the specifics of other answers, but one way to deal with this is to not have the noise (or as much noise) in the source data to begin with.</p>

<p>The noise is coming from the fact that there is high variance in the rendering - the number of samples you've taken haven't converged enough to the actual right answer of the integral, and so some pixels are too high/bright and some are too low/dim (in each color channel).</p>

<p>The problem is this: If you use white noise random numbers to do your sampling, you may get samples clumping together like the image below.  Given enough samples, it will converge, but it will take a while before it gives good coverage over the sampling space.  Find a region of empty space in the image below (like in the lower right) and imagine that there was a small, bright light there and that the scene was dark everywhere else.  You can see how not having any samples there is going to make a problem for rendering.</p>

<p><a href="https://i.sstatic.net/4cQjK.png" rel="noreferrer"><img src="https://i.sstatic.net/4cQjK.png" alt="enter image description here"></a></p>

<p>Alternately, you could sample at even intervals like the below, but that will give you aliasing artifacts instead of noise, which is worse.</p>

<p><a href="https://i.sstatic.net/rHjVE.png" rel="noreferrer"><img src="https://i.sstatic.net/rHjVE.png" alt="enter image description here"></a></p>

<p>One idea is to use low discrepancy sequences and do quasi monte carlo integration (<a href="https://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method" rel="noreferrer">https://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method</a>).  Low discrepancy sequences are related to blue noise, which has only high frequency components.  Going these routes, you get faster convergence of $O(1/N)$ instead of $O(\sqrt{N})$.  These give better coverage of the sample space, but since there is some randomness (or random like qualities) to them, they don't have the aliasing issues that regularly spaced sampling does.</p>

<p>Here is a "jittered grid" where you sample on a grid, but use small random offsets within a cell size.  This was invented by pixar and was under patent for a while but is no longer:
<a href="https://i.sstatic.net/kSYFd.png" rel="noreferrer"><img src="https://i.sstatic.net/kSYFd.png" alt="enter image description here"></a></p>

<p>Here is a common low discrepancy sequence called the Halton sequence (basically a 2d version of Van Der Corpus)</p>

<p><a href="https://i.sstatic.net/Ayirs.png" rel="noreferrer"><img src="https://i.sstatic.net/Ayirs.png" alt="enter image description here"></a></p>

<p>And here is a poisson disc sampling, using Mitchel's best candidate algorithm:<a href="https://i.sstatic.net/DgPeP.png" rel="noreferrer"><img src="https://i.sstatic.net/DgPeP.png" alt="enter image description here"></a></p>

<p>More information, including the source code that generated these images can be found here: <a href="https://blog.demofox.org/2017/05/29/when-random-numbers-are-too-random-low-discrepancy-sequences/" rel="noreferrer">https://blog.demofox.org/2017/05/29/when-random-numbers-are-too-random-low-discrepancy-sequences/</a></p>

CC BY-SA 4.0
2023-12-08T12:43:31.310Z
out of curiosity, when you mention these sampling patterns, you meant the sampling pattern in the `virtual pixel space`, which means when the light rays are started from the `virtual camera` throughout the `virtual image plane`? For example, if I have 6 samples to shoot, those will follow either white/blue/regular uniform patter, right?
bim
CC BY-SA 4.0
2023-12-08T12:50:16.760Z
Or over the hemisphere?
bim
CC BY-SA 3.0
2017-06-22T18:51:45.090Z
2017-06-22T18:57:31.190Z
2017-06-22T18:57:31.190Z
Alan Wolfe
<p>One technique you could use is break the image into blocks and measure each blocks variance - this way you can apply more samples to blocks with higher variance. </p>

<p>The variance can be estimated by using 2 accumulation buffers instead of 1. You render each pass into an alternate buffer. The absolute difference between these buffers (with respect to each block) is proportional to variance. Upon presentation to screen you can add the two buffers together to get your full accumulation buffer back.</p>

CC BY-SA 4.0
2021-03-15T07:44:38.850Z
How would you measure the absolute difference?  Something like suming sqrt((r1-r2)^2 + (g1-g2)^2 + (b1-b2)^2) for all pixels in a block, or summing (abs(r1-r2) + abs(g1-g2) + abs(b1-b2)) for all pixels, or something more advanced?

I've seen (and implemented) methods where you divide the difference in lengths between single pixel values with the sum of the same lengths in two accumulators, and keep sampling that pixel until the ratio is below some limit, but perhaps a smallish patch is better to compare?
alkanen
CC BY-SA 4.0
2021-03-17T03:23:24.237Z
I suppose it's subjective when it comes to how many samples is enough, with squared difference you're going to need a lot more samples to converge fireflies away but the quality will be higher.
PaulHK
CC BY-SA 4.0
2021-03-17T07:39:57.930Z
I would suggest using luminance or some other perceptual metric too. It doesn't need be an average either (fireflies won't stand out much in a 8x8 block average), it could be the max per-pixel difference for a block.
PaulHK
CC BY-SA 3.0
2017-06-23T02:20:29.577Z
2017-06-23T02:47:17.973Z
2017-06-23T02:47:17.973Z
PaulHK
<p>I have been working on my own renderer for a while, and I'm wondering if there's any way to remove the Monte Carlo noise from the rendered image, besides waiting for a long time for it to converge?</p>

<p>The way I found is to blur the image, which is not really helpful, since it reduces the quality/sharpness of the image a lot. And I can achieve the same thing by rendering a small image with more samples, then scaling it up.</p>

<p>Is there any algorithm designed to deal with noise in the image in path tracing?</p>

CC BY-SA 3.0
2017-06-22T23:31:02.323Z
Are you more interested in post processing to disguise the noise, or ways of speeding up convergence so that less noise is present?
trichoplax is on Codidact now
CC BY-SA 3.0
2017-06-23T08:58:25.933Z
FWIW, Benedikt Bitterli recently released the following https://twitter.com/tunabrain/status/872174108385136640   based on his denoising paper.
Simon F
CC BY-SA 3.0
2017-06-23T09:10:57.343Z
In postprocessing area, there is nice algorithm called bilateral filter https://www.shadertoy.com/view/4dfGDH
mdkdy
<algorithm><image-processing><pathtracing><denoise>
Algorithms to Remove High Frequency Noise from Path Tracing
Mary Chang
# Algorithms to Remove High Frequency Noise from Path Tracing

(Asked by: username_0 on 2017-06-22)

I have been working on my own renderer for a while, and I'm wondering if there's any way to remove the Monte Carlo noise from the rendered image, besides waiting for a long time for it to converge?

The way I found is to blur the image, which is not really helpful, since it reduces the quality/sharpness of the image a lot. And I can achieve the same thing by rendering a small image with more samples, then scaling it up.

Is there any algorithm designed to deal with noise in the image in path tracing?

**Tags:** algorithm, image-processing, pathtracing, denoise

> **Comments:**
> **username_1**: Are you more interested in post processing to disguise the noise, or ways of speeding up convergence so that less noise is present?  
> **username_2**: FWIW, Benedikt Bitterli recently released the following https://twitter.com/tunabrain/status/872174108385136640   based on his denoising paper.  
> **username_3**: In postprocessing area, there is nice algorithm called bilateral filter https://www.shadertoy.com/view/4dfGDH  

---

## 2 answers

### Answer by username_4 on 2017-06-22. Score: 8

There are, and I am looking forward to seeing the specifics of other answers, but one way to deal with this is to not have the noise (or as much noise) in the source data to begin with.

The noise is coming from the fact that there is high variance in the rendering - the number of samples you've taken haven't converged enough to the actual right answer of the integral, and so some pixels are too high/bright and some are too low/dim (in each color channel).

The problem is this: If you use white noise random numbers to do your sampling, you may get samples clumping together like the image below. Given enough samples, it will converge, but it will take a while before it gives good coverage over the sampling space. Find a region of empty space in the image below (like in the lower right) and imagine that there was a small, bright light there and that the scene was dark everywhere else. You can see how not having any samples there is going to make a problem for rendering.

[![enter image description here](https://i.sstatic.net/4cQjK.png)](https://i.sstatic.net/4cQjK.png)

Alternately, you could sample at even intervals like the below, but that will give you aliasing artifacts instead of noise, which is worse.

[![enter image description here](https://i.sstatic.net/rHjVE.png)](https://i.sstatic.net/rHjVE.png)

One idea is to use low discrepancy sequences and do quasi monte carlo integration (<https://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method>). Low discrepancy sequences are related to blue noise, which has only high frequency components. Going these routes, you get faster convergence of $O(1/N)$ instead of $O(\sqrt{N})$. These give better coverage of the sample space, but since there is some randomness (or random like qualities) to them, they don't have the aliasing issues that regularly spaced sampling does.

Here is a "jittered grid" where you sample on a grid, but use small random offsets within a cell size. This was invented by pixar and was under patent for a while but is no longer:
[![enter image description here](https://i.sstatic.net/kSYFd.png)](https://i.sstatic.net/kSYFd.png)

Here is a common low discrepancy sequence called the Halton sequence (basically a 2d version of Van Der Corpus)

[![enter image description here](https://i.sstatic.net/Ayirs.png)](https://i.sstatic.net/Ayirs.png)

And here is a poisson disc sampling, using Mitchel's best candidate algorithm:[![enter image description here](https://i.sstatic.net/DgPeP.png)](https://i.sstatic.net/DgPeP.png)

More information, including the source code that generated these images can be found here: <https://blog.demofox.org/2017/05/29/when-random-numbers-are-too-random-low-discrepancy-sequences/>

> **Comments:**
> **username_5**: out of curiosity, when you mention these sampling patterns, you meant the sampling pattern in the `virtual pixel space`, which means when the light rays are started from the `virtual camera` throughout the `virtual image plane`? For example, if I have 6 samples to shoot, those will follow either white/blue/regular uniform patter, right?  
> **username_5**: Or over the hemisphere?  

---

### Answer by username_6 on 2017-06-23. Score: 4

One technique you could use is break the image into blocks and measure each blocks variance - this way you can apply more samples to blocks with higher variance.

The variance can be estimated by using 2 accumulation buffers instead of 1. You render each pass into an alternate buffer. The absolute difference between these buffers (with respect to each block) is proportional to variance. Upon presentation to screen you can add the two buffers together to get your full accumulation buffer back.

> **Comments:**
> **username_7**: How would you measure the absolute difference?  Something like suming sqrt((r1-r2)^2 + (g1-g2)^2 + (b1-b2)^2) for all pixels in a block, or summing (abs(r1-r2) + abs(g1-g2) + abs(b1-b2)) for all pixels, or something more advanced?  I've seen (and implemented) methods where you divide the difference in lengths between single pixel values with the sum of the same lengths in two accumulators, and keep sampling that pixel until the ratio is below some limit, but perhaps a smallish patch is better to compare?  
> **username_6**: I suppose it's subjective when it comes to how many samples is enough, with squared difference you're going to need a lot more samples to converge fireflies away but the quality will be higher.  
> **username_6**: I would suggest using luminance or some other perceptual metric too. It doesn't need be an average either (fireflies won't stand out much in a 8x8 block average), it could be the max per-pixel difference for a block.  

---

