<p>temporal reprojection works surprisingly well and simplistic with (in between) volume-marching (where rays scatter through a cloud of different density, with hard or smooth density-gradients)</p>
<p>validity is is then as simple as an upper bound difference between 2 euclidean-distances (currentFrame + previousFrame)</p>
<p>it may help, to use objectIDs to maintain sharper borders between any 2 solids, but too sharp borders will cause artefacts on movement. The definition of &quot;solid&quot; starts to get fluent, as soon as you do pure volume-marching, and then objectIDs may not be needed for reprojection (and only vor texturing), as in many more basic shaders of
<a href="https://www.shadertoy.com/results?query=reproject&amp;sort=newest&amp;filter=" rel="nofollow noreferrer">https://www.shadertoy.com/results?query=reproject&amp;sort=newest&amp;filter=</a>
explore my volume-marching with quaternion-camera unifications:
<a href="https://www.shadertoy.com/view/7lBXWW" rel="nofollow noreferrer">https://www.shadertoy.com/view/7lBXWW</a> (voumetric laser refraction on spheres+prism)
<a href="https://www.shadertoy.com/view/wl3yDs" rel="nofollow noreferrer">https://www.shadertoy.com/view/wl3yDs</a> (2 dynamic lights near a mandelbulb)</p>

CC BY-SA 4.0
2022-02-20T18:50:47.817Z
2022-02-20T18:50:47.817Z
ollj
<p>One strategy mentioned in <a href="http://advances.realtimerendering.com/s2014/epic/TemporalAA.pptx" rel="noreferrer">Brian Karis's talk about TAA</a> is neighborhood clamping. The general idea is that, for the previous frame's pixel to be valid, its color should be in the color range found in the neighborhood (say 3x3 pixels) of the current pixel this frame.</p>

<p>This rejects history from changing light conditions, which is probably what you want anyway if you don't want moving shadows to produce ghosting.</p>

<p>(Animated textures, depending on the speed of the animation, could also be handled with a motion vector, if you have a predictable UV mapping or can guess reasonably well.)</p>

CC BY-SA 3.0
2015-08-09T14:16:34.443Z
Have you heard of anyone using the depth buffer value as a sanity check?
Alan Wolfe
CC BY-SA 3.0
2015-08-09T16:48:45.367Z
@AlanWolfe No, and I think that is because the motion vector texture is usually 2-component: you'd need a change-in-Z component to know what the depth buffer value should be, and that's not nicely bounded by the screen size. I suspect you could get better rejection strategies than that by adding more per-pixel information.
John Calsbeek
CC BY-SA 3.0
2015-08-09T17:26:05.140Z
Ah OK. What kind of information do you think would be helpful to add. Shading parameter type stuff to be able to tell if it's the same material?
Alan Wolfe
CC BY-SA 3.0
2015-08-09T18:11:28.410Z
@AlanWolfe I don't have a lot of concrete ideas. I'm not an expert on when temporal reprojection with neighborhood clamping breaks down and produces artifacts and what information would be useful in those situations. Perhaps transparents (no motion vector information) combined with high-frequency lighting are producing artifacts, and you need some obscurance information. Perhaps geometric aliasing is your problem and you need some other information.
John Calsbeek
CC BY-SA 3.0
2015-08-09T06:39:28.673Z
2015-08-09T06:39:28.673Z
John Calsbeek
<p>Temporal anti aliasing (and other temporal algorithms) work by matching pixels this frame with pixels from the last frame and then using that information.</p>

<p>I get that you can use the last and current frame matrices along with motion vector information to match the pixels between frames.</p>

<p>What I don't get though is how do you know whether the reprojected pixel is valid or not?  For instance the old pixel may now be hidden behind a different object.</p>

<p>Is it by color only? If so, how are animated textures or changing light conditions handled?</p>

<post-processing>
How does temporal reprojection work?
Alan Wolfe
# How does temporal reprojection work?

(Asked by: username_0 on 2015-08-09)

Temporal anti aliasing (and other temporal algorithms) work by matching pixels this frame with pixels from the last frame and then using that information.

I get that you can use the last and current frame matrices along with motion vector information to match the pixels between frames.

What I don't get though is how do you know whether the reprojected pixel is valid or not? For instance the old pixel may now be hidden behind a different object.

Is it by color only? If so, how are animated textures or changing light conditions handled?

**Tags:** post-processing


---

## 2 answers

### Answer by username_2 on 2015-08-09. Score: 6 | Accepted answer

One strategy mentioned in [Brian Karis's talk about TAA](http://advances.realtimerendering.com/s2014/epic/TemporalAA.pptx) is neighborhood clamping. The general idea is that, for the previous frame's pixel to be valid, its color should be in the color range found in the neighborhood (say 3x3 pixels) of the current pixel this frame.

This rejects history from changing light conditions, which is probably what you want anyway if you don't want moving shadows to produce ghosting.

(Animated textures, depending on the speed of the animation, could also be handled with a motion vector, if you have a predictable UV mapping or can guess reasonably well.)

> **Comments:**
> **username_0**: Have you heard of anyone using the depth buffer value as a sanity check?  
> **username_2**: @username_0 No, and I think that is because the motion vector texture is usually 2-component: you'd need a change-in-Z component to know what the depth buffer value should be, and that's not nicely bounded by the screen size. I suspect you could get better rejection strategies than that by adding more per-pixel information.  
> **username_0**: Ah OK. What kind of information do you think would be helpful to add. Shading parameter type stuff to be able to tell if it's the same material?  
> **username_2**: @username_0 I don't have a lot of concrete ideas. I'm not an expert on when temporal reprojection with neighborhood clamping breaks down and produces artifacts and what information would be useful in those situations. Perhaps transparents (no motion vector information) combined with high-frequency lighting are producing artifacts, and you need some obscurance information. Perhaps geometric aliasing is your problem and you need some other information.  

---

### Answer by username_1 on 2022-02-20. Score: 1

temporal reprojection works surprisingly well and simplistic with (in between) volume-marching (where rays scatter through a cloud of different density, with hard or smooth density-gradients)

validity is is then as simple as an upper bound difference between 2 euclidean-distances (currentFrame + previousFrame)

it may help, to use objectIDs to maintain sharper borders between any 2 solids, but too sharp borders will cause artefacts on movement. The definition of "solid" starts to get fluent, as soon as you do pure volume-marching, and then objectIDs may not be needed for reprojection (and only vor texturing), as in many more basic shaders of
<https://www.shadertoy.com/results?query=reproject&sort=newest&filter=>
explore my volume-marching with quaternion-camera unifications:
<https://www.shadertoy.com/view/7lBXWW> (voumetric laser refraction on spheres+prism)
<https://www.shadertoy.com/view/wl3yDs> (2 dynamic lights near a mandelbulb)


---

