<p>Ok. So finally, I have solved the problem. This problem I been working on for a couple of weeks. Had to go through several books in math, game devevelopment, etc. Went through all my code, to see if I had something I did misunderstand.</p>
<p>Finally I found the problem! I was trying to store normals in the texture, with values between [-1, 1], I did not think about the values stored in the texture has to be [0, 1].</p>
<p>So I normalized them to [0, 1]. And then in the second pass I the normalize them back to [-1, 1], and finally, after many long nights of search, the problem has been found!</p>
<p>Hope this will help others also! And thanks everyone who had a look!</p>
<p>SOLVED!</p>

CC BY-SA 4.0
2020-09-17T15:08:08.677Z
sometimes these very tiny things cost the most time to find... But nice, that you found it.
Thomas
CC BY-SA 4.0
2020-09-17T15:10:51.000Z
They really do!
But I learned _alot_ from having this problem. I was forced to read up on alot of stuff on the way
the1bird
CC BY-SA 4.0
2020-09-16T20:23:31.983Z
2024-05-29T12:46:21.310Z
2024-05-29T12:46:21.310Z
the1bird
<p>You're not correctly inverting the mapping you used to store the depth value. When storing it you use:</p>
<pre><code>vDepth = -(vPos.z - NearPlane) / ( FarPlane - NearPlane);
</code></pre>
<p>Then you reconstruct it as:</p>
<pre><code>const float depth = normalDepth.w * (FarPlane - NearPlane);
</code></pre>
<p>but this misses both the offset by NearPlane, and the negative sign to flip the Z axis (since it points out of the screen in right-handed view space, but you want to store positive values). You're not getting back the same view space you started from—the space that your normals and light are in.</p>
<p>The correct inversion should be:</p>
<pre><code>const float depth = -(normalDepth.w * (FarPlane - NearPlane) + NearPlane);
</code></pre>
<p>The depth values will be negative; this is correct since as mentioned Z points out of the screen in right-handed view space, so the Z values of everything you can see will be below zero. You'll have to tweak some other bits of your code to match—for instance, get rid of the negative sign when you calculate the position xy, and move the light to a negative Z value. It's this attempting to work in a space where depths are positive, while using normals in a space where depth is negative, that is most likely causing the lighting issues when rotating.</p>
<p>Your unprojection looks like it's missing the fact that horizontal and vertical FOVs are not the same. You need the aspect ratio in there (same way as used to calculate the projection matrix). In fact, you can just use the scale factors from the projection matrix.</p>
<pre><code>vec3 position = vec3(vDeviceCoords.x / ProjectionMatrix[0][0], vDeviceCoords.y / ProjectionMatrix[1][1], 1.0);
position *= depth;
</code></pre>
<p>You could also precalculate those 1/ProjectionMatrix values in a separate uniform value if you liked.</p>
<p>Finally, I'll just note that you don't need to store a depth value explicitly in a channel of a framebuffer somewhere. It's possible to bind the actual depth buffer as a texture and read that. It's stored using a nonlinear mapping so it takes a bit more code to recover a linear depth value from it, but this ends up being the best way as you get the full precision of the hardware depth buffer, and it doesn't consume a framebuffer channel that you could use for something else. For more information, see <a href="https://mynameismjp.wordpress.com/2010/09/05/position-from-depth-3/" rel="nofollow noreferrer">MJP's Position from Depth 3 post</a>.</p>

CC BY-SA 4.0
2020-09-15T17:47:32.727Z
Thank you for your reply Nathan!

I agree I was off with the nearplane (Note: Justing nearplane = 1).

I think the depth value was stored with correct sign? "vDepth = -(vPos.z - NearPlane) / ( FarPlane - NearPlane);". You maybe missed the first minus sign there? (anyways did try exactly what you said).


Anyways, I been trying what you said now, I get same result :(!

I think only part of what you wrote that I did not get was "and move the light to a negative Z value"

What do you mean by that? It is a stationary light, I did put it in a place where I want it?
the1bird
CC BY-SA 4.0
2020-09-15T17:50:36.637Z
I been studying the method posted in the blog "MJP's Position from Depth 3 post.".
I try to do like him, reconstruct the viewspace position from the stored viewspace depthvalue. I cant see what the difference is. But he do not calculate any light in viewspace, dunno if that is something I am failing on somehow. Or maybe I am just crazy, certently starting to feel crazy....
the1bird
CC BY-SA 4.0
2020-09-15T17:52:44.543Z
The depth value was stored with the correct sign, but then reconstructed with the wrong sign, is what I'm saying. The original sign is not getting restored by the code you posted.
Nathan Reed
CC BY-SA 4.0
2020-09-15T17:55:27.587Z
Re: "Move the light to a negative Z-value" never mind that, I missed that the light position is in world space not view space. In any case, after transforming by the ViewMatrix it should end up with a negative Z value since it's in front of the camera.
Nathan Reed
CC BY-SA 4.0
2020-09-15T17:59:05.800Z
Okok. Hmm does not help so far =/. Light still rotating aound
the1bird
CC BY-SA 4.0
2020-09-15T17:59:09.867Z
BTW for debugging, try visualizing the XYZ positions as RGB, by setting your shaders to output `fract(vPos)` to see the view position from the first pass, or `fract(position)` to see the reconstructed position in the second pass. The results from those two images should match.
Nathan Reed
CC BY-SA 4.0
2020-09-15T18:16:53.767Z
Much love! You dont know hos much i been trying. Will do in a moment! And post screen
the1bird
CC BY-SA 4.0
2020-09-15T18:37:52.487Z
Not sure how to look at that. But I guess it looks wrong? Not sure.

I added screenshots above
the1bird
CC BY-SA 4.0
2020-09-15T19:42:27.103Z
OK. So you've got your fract(position) from the second pass, now what you need to do is write out fract(vPos) from the first pass and compare that. The first pass vPos is the ground truth. The second pass position needs to be made to match that.
Nathan Reed
CC BY-SA 4.0
2020-09-15T20:21:48.917Z
I added GIF's with that, but dont know how to go forward from here, I am running out of ideas
the1bird
CC BY-SA 4.0
2020-09-15T16:34:21.613Z
2020-09-15T16:55:12.223Z
2020-09-15T16:55:12.223Z
Nathan Reed
<p><strong>Current state:</strong></p>
<p>I believe the reconstructed positions are correct. However, I still have the same problem as all the time. When I rotate around my single pointlight appears to be moving around also.</p>
<p>I finally solved this! It was annoying, but I did learn a lot on the way.</p>
<hr />
<p><strong>The explanation</strong></p>
<p>I am trying to recreate viewspace positions from only a stored depth value, and then calculate the light in viewspace.</p>
<p>In the first pass I store the Normals and the normalized depth in viewspace.
In the second pass I try to build reconstruct the position in viewspace.</p>
<p>I believe something is going wrong in the reconstruction, but I can't see what, or if it something else. I did try to reconstruct the position in different ways, always with the same not good result.</p>
<p>The light initially looks fine, but when I start moving around the light seems to be moving around.
When I go further away, things get less lit. When rotating the light also rotates.
Note: When I render out the actual depth values, it looks fine, so think the depth is fine.</p>
<p>What could I be doing wrong?</p>
<p>Below is code for the fragment &amp; vertex shader, first and second pass.</p>
<p><strong>First pass:</strong></p>
<pre><code>--Vertex
#version 460

layout( location = 0 ) in vec3 VertexPosition;
layout( location = 1 ) in vec3 VertexNormal;

/* Uniforms */
uniform mat4 ViewMatrix;
uniform mat4 ProjectionMatrix;
uniform mat3 NormalMatrix;
uniform float FarPlane;
uniform float NearPlane;

out vec3 vNormal;
out highp float vDepth;

void main() {
  mat4 vMVP = ProjectionMatrix * ViewMatrix;
  gl_Position = vMVP * vec4( VertexPosition, 1.0 ); // Screenspace

  vNormal = normalize(NormalMatrix  * normalize(VertexNormal));
  vec3 vPos = (ViewMatrix * vec4( VertexPosition, 1.0 )).xyz;

  // Save normalized depth [0, 1]
  vDepth = -(vPos.z - NearPlane) / ( FarPlane - NearPlane);
}

--Fragment
#version 460

in vec3 vNormal;
in highp float vDepth;

layout (location = 0) out highp vec4 NormalDepth;

void main() {
  NormalDepth = vec4(normalize(vNormal), vDepth);
}
</code></pre>
<p><strong>Second pass</strong> (off rendering to screen):</p>
<pre><code>--Vertex
#version 460

layout( location = 0 ) in vec3 VertexPosition;

out vec2 vTexCoord;
out vec2 vDeviceCoords;

void main() {
  vDeviceCoords = 2.0 * ( VertexPosition.xy - 0.5 );// [-1, 1]
  vTexCoord = VertexPosition.xy;//  [0, 1] // Just 2 triangles for the screen

  gl_Position = vec4(vDeviceCoords, 1.0, 1.0 ); // [-1, 1]
}

--Fragment
#version 460

/* Uniforms */
uniform float NearPlane;
uniform float FarPlane;
uniform mat4 ViewMatrix;

in vec2 vTexCoord;
in vec2 vDeviceCoords;

layout( location = 0 ) out vec4 FragColor;
layout( binding = 0 ) uniform highp sampler2D NormalDepthTex;

vec3 diffuseValue( vec3 normal, vec3 position)
{
  // Light position. Here now for testing purposes
  // Stationary point light, transformed to viewspace
  vec3 vLightPos = (ViewMatrix * vec4( -10.0, 0.0, 2.0, 1.0 )).xyz;

  normal = normalize(normal);
  const vec3 lightDir = normalize(position - vLightPos);
  const float diffuseCos = max( dot( normal, -lightDir ), 0.0 );

  return vec3(diffuseCos);
}

void main() {
  const vec4 normalDepth = texture2D( NormalDepthTex, vTexCoord );

  // Normalize depth to [0, FarPlane]
  const float depth = normalDepth.w * (FarPlane - NearPlane);

  // Unproject
  vec3 position;
  position = vec3(vDeviceCoords.x, vDeviceCoords.y, NearPlane);
  position.xy *= -depth / NearPlane; 
  position.z = depth;

  // View space
  vec3 diffuseValue = diffuseValue(normalDepth.xyz, position);

  FragColor = vec4( diffuseValue.x, diffuseValue.y, diffuseValue.z, 1.0 );
}
</code></pre>
<p><strong>Edit 1</strong></p>
<p>Adding screenshot.</p>
<p>For the normals, they looks correct, always pointing toward the camera. (Also I can add, if I do the light in the first pass (not reconstructed depth), then the light is 100% fine).</p>
<p><a href="https://i.sstatic.net/ts4j7.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ts4j7.png" alt="Rendering the normals" /></a></p>
<p>If we look at the light, initially it looks good. I positioned the light so It should shade as seen. But it changes and becomes wrong when I rote the camera. Also the intensity changes when I go further/closer with the camera.</p>
<p><a href="https://i.sstatic.net/63LmJ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/63LmJ.png" alt="The light (albedo value)" /></a></p>
<p>If we slightly rotate left for example, then the light follows (I want it in position!):</p>
<p><a href="https://i.sstatic.net/NUm2I.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/NUm2I.png" alt="Light moved" /></a></p>
<p>The viewspace depth [0, 1]</p>
<p><a href="https://i.sstatic.net/BjdEy.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/BjdEy.png" alt="enter image description here" /></a></p>
<p><strong>EDIT 2:</strong></p>
<p>I have now reconstructed the depth in three different ways, all with the same result. I am starting to think There is something else I am doing wrong? Do I calculate the light wrong or something?</p>
<p>The three ways I have rebuilt (or tried to) are:</p>
<p><em>Option 1:</em></p>
<pre><code>    depth  = normalDepth.w; // Should be [0, 1]

    // Create ray to FarPlane
    // HalfVec is FarPlane * glm::tan( glm::radians( m_fov / 2.0f ) )
    vec3 ray = vec3(vDeviceCoords * HalfVec, FarPlane);

    // Unprojection
    position.xy = ray.xy * -depth / NearPlane;
    position.z = depth * FarPlane;
</code></pre>
<p><em>Option 2:</em></p>
<pre><code>    depth  = normalDepth.w * FarPlane;

    vec2 p = vDeviceCoords * NearPlane * tan(radians(Fov / 2.0));
    vec2 p_ = (p / NearPlane) * -depth;

   position = vec3(p_, depth);
</code></pre>
<p><em>Option 3:</em></p>
<pre><code>    depth  = normalDepth.z;

    mat4 inversePrjMat = inverse( ProjectionMatrix );
    vec4 viewPosH      = inversePrjMat * vec4( vDeviceCoords.x, vDeviceCoords.y, 1.0, 1.0 );
    vec3 farPlaneRay   = viewPosH.xyz / viewPosH.w;

    position = farPlaneRay;
    position.xy *= -depth;
    position.z = depth * FarPlane;
</code></pre>
<p>These options all end up the same. I have recalculated and gone through the math over and over again, there is something else I am missing is my feeling.</p>
<p><strong>Edit 3</strong></p>
<p>I was asked to upload image with the normals (R, G, B) when camera is a bit rotated.</p>
<p><a href="https://i.sstatic.net/AgUzI.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/AgUzI.png" alt="enter image description here" /></a></p>
<p><strong>Edit 4</strong></p>
<p>Adding some GIF animations that shows what is going on with the light. As you can see, if I don't rotate at all, just move around, then the light is as it should.</p>
<p><a href="https://i.sstatic.net/RQGjW.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/RQGjW.gif" alt="Moving around 1" /></a></p>
<p><a href="https://i.sstatic.net/G3reu.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/G3reu.gif" alt="Moving around 2" /></a></p>
<p><a href="https://i.sstatic.net/DzBFa.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/DzBFa.gif" alt="Moving around 3" /></a></p>
<p><a href="https://i.sstatic.net/k0Xds.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/k0Xds.gif" alt="Moving around 4" /></a></p>
<p><strong>Edit 5</strong></p>
<p>I did the correction  Nathan Reed pointed out. Still with no better result.
This is what it looks like when I render out fract(position).
The boxes I render have a size 5.0 * 5.0 * 5.0 (actually each box is made up of 5x5x5 smaller boxes)</p>
<p><a href="https://i.sstatic.net/qg865.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/qg865.jpg" alt="fract(position)" /></a></p>
<p>and a bit more angled view at the boxes</p>
<p><a href="https://i.sstatic.net/s1lxj.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/s1lxj.jpg" alt="fract(position) with angle" /></a></p>
<p><strong>Edit 6</strong></p>
<p>This is frontview, rendering <code>vec4( fract(position.x), 0.0, 0.0, 1.0 );</code> <a href="https://i.sstatic.net/5l74u.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/5l74u.png" alt="vec4( fract(position.x), 0.0, 0.0, 1.0 )" /></a></p>
<p>This is with  vec4( fract(position.y), 0.0, 0.0, 1.0 )</p>
<p><a href="https://i.sstatic.net/DXys6.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/DXys6.png" alt=" vec4( fract(position.y), 0.0, 0.0, 1.0 )" /></a></p>
<p>This is what it looks like when i render  vec4( VertexPosition, 1.0 ); (the 2 triangles making up the fullscreen in the second pass)</p>
<p><a href="https://i.sstatic.net/rksek.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/rksek.png" alt="enter image description here" /></a></p>
<p><strong>edit 7</strong></p>
<p>This is what is looks like if i render the fract(vPos) from the first pass.
<a href="https://i.sstatic.net/z3CJl.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/z3CJl.gif" alt="fract(vPos) from the first pass" /></a></p>
<p>Here I render just vPos from the first pass
<a href="https://i.sstatic.net/w7o1w.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/w7o1w.gif" alt="enter image description here" /></a></p>
<p><strong>Edit 8</strong></p>
<p>This is about how it should look like. This is when I do the light calculation directly in the first pass. It is stationary, and perfect, as it should be, in this case.</p>
<p><a href="https://i.sstatic.net/ss3y7.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ss3y7.png" alt="enter image description here" /></a></p>
<p><strong>Edit 9</strong></p>
<p>I just rendered the constructed positions, and noticed x,y-axes are flipped.
I need to negate x,y when multiplying with depth, since depth is negative.
Also we are saving the depth in viewspace, so use it straight for the z-component. That is correct, it can be noticed by the light not fading away when moving away further with the position.z set to depth.
Also, we are multiplying xy with depth since triangle similarity. To be more precise we should divide that with the nearplane, I will do that, but I am using nearplane = 1, so won't be any practical difference right now anyways.</p>
<p>So I did this:</p>
<pre><code>    position = vec3(vDeviceCoords.x / ProjectionMatrix[0][0], vDeviceCoords.y / ProjectionMatrix[1][1], 1.0);
    position.xy *= -depth / NearPlane;
    position.z  = depth;
</code></pre>
<p>Now the light calculated on the second pass looks very much correct, apart from rotating... The lights don't even fade when going further. As long as I don't rotate (!)</p>
<p>Is there something else obvious I am missing?</p>
<p>Light, when moved, but not rotated at all:</p>
<p><a href="https://i.sstatic.net/g3zm1.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/g3zm1.png" alt="Moved, but rotated" /></a></p>
<p>Reconstructed and Position looks the same when rendered. but when doing fract on the reconstructed position I still get this:</p>
<p><a href="https://i.sstatic.net/uMn0s.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/uMn0s.jpg" alt="enter image description here" /></a></p>
<p><strong>Edit 10</strong></p>
<p>Still not giving up, learning a lot, but getting a bit crazy with this problem.
So now when I look at the reconstructed using <em>fract(position)</em>. This below is what I get (right side is the fract part, left side is just the scene with diffuse point light moving around wrongly).</p>
<p>The position seems to be correct now when I look at it, as each texture on left side is one unit. And that is matching the reconstructed position:</p>
<p><a href="https://i.sstatic.net/OGmFp.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/OGmFp.jpg" alt="enter image description here" /></a></p>
<p>However, as Nathan pointed out. I can look at <em>fract(vPos)</em> in the fist pass, that is how the reconstructed position also should look.</p>
<p>And still, this is how it looks like, from the first pass:</p>
<p><a href="https://i.sstatic.net/tEtux.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/tEtux.png" alt="enter image description here" /></a></p>
<p>That one makes less sense for me. Should I not be able to see the fractions in the same way there? For me the reconstructed one makes more sense.</p>
<p><strong>Edit 11</strong></p>
<p>New findings!</p>
<p>I did fract directly in the first pass, to see how it looks like. I did not think about the values getting changed when saved to texture, so I was not looking at what I though before.</p>
<p>Anyways, it looks exactly the same as the reconstructed values, see below!</p>
<p><a href="https://i.sstatic.net/GQvMR.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/GQvMR.jpg" alt="enter image description here" /></a></p>
<p>So I believe now I have the right constructed depth. But however, still having the same issues as always, light seems to rotate around when I am moving.</p>

CC BY-SA 4.0
2020-09-15T15:19:12.287Z
can you also add an image of the normals when you make this small rotation? I think this could help understanding the problem
Thomas
CC BY-SA 4.0
2020-09-17T14:39:37.600Z
in the first shader (vertex) what is the normalMatrix? is it the transpose(inverse(modelViewMatrix))? If not, maybe this is your problem
Thomas
<opengl><shader><glsl><lighting>
OpenGL - Recreate position in viewspace from depth values
the1bird
# OpenGL - Recreate position in viewspace from depth values

(Asked by: username_0 on 2020-09-14)

**Current state:**

I believe the reconstructed positions are correct. However, I still have the same problem as all the time. When I rotate around my single pointlight appears to be moving around also.

I finally solved this! It was annoying, but I did learn a lot on the way.

---

**The explanation**

I am trying to recreate viewspace positions from only a stored depth value, and then calculate the light in viewspace.

In the first pass I store the Normals and the normalized depth in viewspace.
In the second pass I try to build reconstruct the position in viewspace.

I believe something is going wrong in the reconstruction, but I can't see what, or if it something else. I did try to reconstruct the position in different ways, always with the same not good result.

The light initially looks fine, but when I start moving around the light seems to be moving around.
When I go further away, things get less lit. When rotating the light also rotates.
Note: When I render out the actual depth values, it looks fine, so think the depth is fine.

What could I be doing wrong?

Below is code for the fragment & vertex shader, first and second pass.

**First pass:**

```
--Vertex
#version 460

layout( location = 0 ) in vec3 VertexPosition;
layout( location = 1 ) in vec3 VertexNormal;

/* Uniforms */
uniform mat4 ViewMatrix;
uniform mat4 ProjectionMatrix;
uniform mat3 NormalMatrix;
uniform float FarPlane;
uniform float NearPlane;

out vec3 vNormal;
out highp float vDepth;

void main() {
  mat4 vMVP = ProjectionMatrix * ViewMatrix;
  gl_Position = vMVP * vec4( VertexPosition, 1.0 ); // Screenspace

  vNormal = normalize(NormalMatrix  * normalize(VertexNormal));
  vec3 vPos = (ViewMatrix * vec4( VertexPosition, 1.0 )).xyz;

  // Save normalized depth [0, 1]
  vDepth = -(vPos.z - NearPlane) / ( FarPlane - NearPlane);
}

--Fragment
#version 460

in vec3 vNormal;
in highp float vDepth;

layout (location = 0) out highp vec4 NormalDepth;

void main() {
  NormalDepth = vec4(normalize(vNormal), vDepth);
}

```

**Second pass** (off rendering to screen):

```
--Vertex
#version 460

layout( location = 0 ) in vec3 VertexPosition;

out vec2 vTexCoord;
out vec2 vDeviceCoords;

void main() {
  vDeviceCoords = 2.0 * ( VertexPosition.xy - 0.5 );// [-1, 1]
  vTexCoord = VertexPosition.xy;//  [0, 1] // Just 2 triangles for the screen

  gl_Position = vec4(vDeviceCoords, 1.0, 1.0 ); // [-1, 1]
}

--Fragment
#version 460

/* Uniforms */
uniform float NearPlane;
uniform float FarPlane;
uniform mat4 ViewMatrix;

in vec2 vTexCoord;
in vec2 vDeviceCoords;

layout( location = 0 ) out vec4 FragColor;
layout( binding = 0 ) uniform highp sampler2D NormalDepthTex;

vec3 diffuseValue( vec3 normal, vec3 position)
{
  // Light position. Here now for testing purposes
  // Stationary point light, transformed to viewspace
  vec3 vLightPos = (ViewMatrix * vec4( -10.0, 0.0, 2.0, 1.0 )).xyz;

  normal = normalize(normal);
  const vec3 lightDir = normalize(position - vLightPos);
  const float diffuseCos = max( dot( normal, -lightDir ), 0.0 );

  return vec3(diffuseCos);
}

void main() {
  const vec4 normalDepth = texture2D( NormalDepthTex, vTexCoord );

  // Normalize depth to [0, FarPlane]
  const float depth = normalDepth.w * (FarPlane - NearPlane);

  // Unproject
  vec3 position;
  position = vec3(vDeviceCoords.x, vDeviceCoords.y, NearPlane);
  position.xy *= -depth / NearPlane; 
  position.z = depth;

  // View space
  vec3 diffuseValue = diffuseValue(normalDepth.xyz, position);

  FragColor = vec4( diffuseValue.x, diffuseValue.y, diffuseValue.z, 1.0 );
}

```

**Edit 1**

Adding screenshot.

For the normals, they looks correct, always pointing toward the camera. (Also I can add, if I do the light in the first pass (not reconstructed depth), then the light is 100% fine).

[![Rendering the normals](https://i.sstatic.net/ts4j7.png)](https://i.sstatic.net/ts4j7.png)

If we look at the light, initially it looks good. I positioned the light so It should shade as seen. But it changes and becomes wrong when I rote the camera. Also the intensity changes when I go further/closer with the camera.

[![The light (albedo value)](https://i.sstatic.net/63LmJ.png)](https://i.sstatic.net/63LmJ.png)

If we slightly rotate left for example, then the light follows (I want it in position!):

[![Light moved](https://i.sstatic.net/NUm2I.png)](https://i.sstatic.net/NUm2I.png)

The viewspace depth [0, 1]

[![enter image description here](https://i.sstatic.net/BjdEy.png)](https://i.sstatic.net/BjdEy.png)

**EDIT 2:**

I have now reconstructed the depth in three different ways, all with the same result. I am starting to think There is something else I am doing wrong? Do I calculate the light wrong or something?

The three ways I have rebuilt (or tried to) are:

*Option 1:*

```
    depth  = normalDepth.w; // Should be [0, 1]

    // Create ray to FarPlane
    // HalfVec is FarPlane * glm::tan( glm::radians( m_fov / 2.0f ) )
    vec3 ray = vec3(vDeviceCoords * HalfVec, FarPlane);

    // Unprojection
    position.xy = ray.xy * -depth / NearPlane;
    position.z = depth * FarPlane;

```

*Option 2:*

```
    depth  = normalDepth.w * FarPlane;

    vec2 p = vDeviceCoords * NearPlane * tan(radians(Fov / 2.0));
    vec2 p_ = (p / NearPlane) * -depth;

   position = vec3(p_, depth);

```

*Option 3:*

```
    depth  = normalDepth.z;

    mat4 inversePrjMat = inverse( ProjectionMatrix );
    vec4 viewPosH      = inversePrjMat * vec4( vDeviceCoords.x, vDeviceCoords.y, 1.0, 1.0 );
    vec3 farPlaneRay   = viewPosH.xyz / viewPosH.w;

    position = farPlaneRay;
    position.xy *= -depth;
    position.z = depth * FarPlane;

```

These options all end up the same. I have recalculated and gone through the math over and over again, there is something else I am missing is my feeling.

**Edit 3**

I was asked to upload image with the normals (R, G, B) when camera is a bit rotated.

[![enter image description here](https://i.sstatic.net/AgUzI.png)](https://i.sstatic.net/AgUzI.png)

**Edit 4**

Adding some GIF animations that shows what is going on with the light. As you can see, if I don't rotate at all, just move around, then the light is as it should.

[![Moving around 1](https://i.sstatic.net/RQGjW.gif)](https://i.sstatic.net/RQGjW.gif)

[![Moving around 2](https://i.sstatic.net/G3reu.gif)](https://i.sstatic.net/G3reu.gif)

[![Moving around 3](https://i.sstatic.net/DzBFa.gif)](https://i.sstatic.net/DzBFa.gif)

[![Moving around 4](https://i.sstatic.net/k0Xds.gif)](https://i.sstatic.net/k0Xds.gif)

**Edit 5**

I did the correction username_2 username_2 pointed out. Still with no better result.
This is what it looks like when I render out fract(position).
The boxes I render have a size 5.0 \* 5.0 \* 5.0 (actually each box is made up of 5x5x5 smaller boxes)

[![fract(position)](https://i.sstatic.net/qg865.jpg)](https://i.sstatic.net/qg865.jpg)

and a bit more angled view at the boxes

[![fract(position) with angle](https://i.sstatic.net/s1lxj.jpg)](https://i.sstatic.net/s1lxj.jpg)

**Edit 6**

This is frontview, rendering `vec4( fract(position.x), 0.0, 0.0, 1.0 );` [![vec4( fract(position.x), 0.0, 0.0, 1.0 )](https://i.sstatic.net/5l74u.png)](https://i.sstatic.net/5l74u.png)

This is with vec4( fract(position.y), 0.0, 0.0, 1.0 )

[![ vec4( fract(position.y), 0.0, 0.0, 1.0 )](https://i.sstatic.net/DXys6.png)](https://i.sstatic.net/DXys6.png)

This is what it looks like when i render vec4( VertexPosition, 1.0 ); (the 2 triangles making up the fullscreen in the second pass)

[![enter image description here](https://i.sstatic.net/rksek.png)](https://i.sstatic.net/rksek.png)

**edit 7**

This is what is looks like if i render the fract(vPos) from the first pass.
[![fract(vPos) from the first pass](https://i.sstatic.net/z3CJl.gif)](https://i.sstatic.net/z3CJl.gif)

Here I render just vPos from the first pass
[![enter image description here](https://i.sstatic.net/w7o1w.gif)](https://i.sstatic.net/w7o1w.gif)

**Edit 8**

This is about how it should look like. This is when I do the light calculation directly in the first pass. It is stationary, and perfect, as it should be, in this case.

[![enter image description here](https://i.sstatic.net/ss3y7.png)](https://i.sstatic.net/ss3y7.png)

**Edit 9**

I just rendered the constructed positions, and noticed x,y-axes are flipped.
I need to negate x,y when multiplying with depth, since depth is negative.
Also we are saving the depth in viewspace, so use it straight for the z-component. That is correct, it can be noticed by the light not fading away when moving away further with the position.z set to depth.
Also, we are multiplying xy with depth since triangle similarity. To be more precise we should divide that with the nearplane, I will do that, but I am using nearplane = 1, so won't be any practical difference right now anyways.

So I did this:

```
    position = vec3(vDeviceCoords.x / ProjectionMatrix[0][0], vDeviceCoords.y / ProjectionMatrix[1][1], 1.0);
    position.xy *= -depth / NearPlane;
    position.z  = depth;

```

Now the light calculated on the second pass looks very much correct, apart from rotating... The lights don't even fade when going further. As long as I don't rotate (!)

Is there something else obvious I am missing?

Light, when moved, but not rotated at all:

[![Moved, but rotated](https://i.sstatic.net/g3zm1.png)](https://i.sstatic.net/g3zm1.png)

Reconstructed and Position looks the same when rendered. but when doing fract on the reconstructed position I still get this:

[![enter image description here](https://i.sstatic.net/uMn0s.jpg)](https://i.sstatic.net/uMn0s.jpg)

**Edit 10**

Still not giving up, learning a lot, but getting a bit crazy with this problem.
So now when I look at the reconstructed using *fract(position)*. This below is what I get (right side is the fract part, left side is just the scene with diffuse point light moving around wrongly).

The position seems to be correct now when I look at it, as each texture on left side is one unit. And that is matching the reconstructed position:

[![enter image description here](https://i.sstatic.net/OGmFp.jpg)](https://i.sstatic.net/OGmFp.jpg)

However, as username_2 pointed out. I can look at *fract(vPos)* in the fist pass, that is how the reconstructed position also should look.

And still, this is how it looks like, from the first pass:

[![enter image description here](https://i.sstatic.net/tEtux.png)](https://i.sstatic.net/tEtux.png)

That one makes less sense for me. Should I not be able to see the fractions in the same way there? For me the reconstructed one makes more sense.

**Edit 11**

New findings!

I did fract directly in the first pass, to see how it looks like. I did not think about the values getting changed when saved to texture, so I was not looking at what I though before.

Anyways, it looks exactly the same as the reconstructed values, see below!

[![enter image description here](https://i.sstatic.net/GQvMR.jpg)](https://i.sstatic.net/GQvMR.jpg)

So I believe now I have the right constructed depth. But however, still having the same issues as always, light seems to rotate around when I am moving.

**Tags:** opengl, shader, glsl, lighting

> **Comments:**
> **username_1**: can you also add an image of the normals when you make this small rotation? I think this could help understanding the problem  
> **username_1**: in the first shader (vertex) what is the normalMatrix? is it the transpose(inverse(modelViewMatrix))? If not, maybe this is your problem  

---

## 2 answers

### Answer by username_0 on 2020-09-16. Score: 1 | Accepted answer

Ok. So finally, I have solved the problem. This problem I been working on for a couple of weeks. Had to go through several books in math, game devevelopment, etc. Went through all my code, to see if I had something I did misunderstand.

Finally I found the problem! I was trying to store normals in the texture, with values between [-1, 1], I did not think about the values stored in the texture has to be [0, 1].

So I normalized them to [0, 1]. And then in the second pass I the normalize them back to [-1, 1], and finally, after many long nights of search, the problem has been found!

Hope this will help others also! And thanks everyone who had a look!

SOLVED!

> **Comments:**
> **username_1**: sometimes these very tiny things cost the most time to find... But nice, that you found it.  
> **username_0**: They really do! But I learned _alot_ from having this problem. I was forced to read up on alot of stuff on the way  

---

### Answer by username_2 on 2020-09-15. Score: 1

You're not correctly inverting the mapping you used to store the depth value. When storing it you use:

```
vDepth = -(vPos.z - NearPlane) / ( FarPlane - NearPlane);

```

Then you reconstruct it as:

```
const float depth = normalDepth.w * (FarPlane - NearPlane);

```

but this misses both the offset by NearPlane, and the negative sign to flip the Z axis (since it points out of the screen in right-handed view space, but you want to store positive values). You're not getting back the same view space you started from—the space that your normals and light are in.

The correct inversion should be:

```
const float depth = -(normalDepth.w * (FarPlane - NearPlane) + NearPlane);

```

The depth values will be negative; this is correct since as mentioned Z points out of the screen in right-handed view space, so the Z values of everything you can see will be below zero. You'll have to tweak some other bits of your code to match—for instance, get rid of the negative sign when you calculate the position xy, and move the light to a negative Z value. It's this attempting to work in a space where depths are positive, while using normals in a space where depth is negative, that is most likely causing the lighting issues when rotating.

Your unprojection looks like it's missing the fact that horizontal and vertical FOVs are not the same. You need the aspect ratio in there (same way as used to calculate the projection matrix). In fact, you can just use the scale factors from the projection matrix.

```
vec3 position = vec3(vDeviceCoords.x / ProjectionMatrix[0][0], vDeviceCoords.y / ProjectionMatrix[1][1], 1.0);
position *= depth;

```

You could also precalculate those 1/ProjectionMatrix values in a separate uniform value if you liked.

Finally, I'll just note that you don't need to store a depth value explicitly in a channel of a framebuffer somewhere. It's possible to bind the actual depth buffer as a texture and read that. It's stored using a nonlinear mapping so it takes a bit more code to recover a linear depth value from it, but this ends up being the best way as you get the full precision of the hardware depth buffer, and it doesn't consume a framebuffer channel that you could use for something else. For more information, see [MJP's Position from Depth 3 post](https://mynameismjp.wordpress.com/2010/09/05/position-from-depth-3/).

> **Comments:**
> **username_0**: Thank you for your reply username_2!  I agree I was off with the nearplane (Note: Justing nearplane = 1).  I think the depth value was stored with correct sign? "vDepth = -(vPos.z - NearPlane) / ( FarPlane - NearPlane);". You maybe missed the first minus sign there? (anyways did try exactly what you said).   Anyways, I been trying what you said now, I get same result :(!  I think only part of what you wrote that I did not get was "and move the light to a negative Z value"  What do you mean by that? It is a stationary light, I did put it in a place where I want it?  
> **username_0**: I been studying the method posted in the blog "MJP's Position from Depth 3 post.". I try to do like him, reconstruct the viewspace position from the stored viewspace depthvalue. I cant see what the difference is. But he do not calculate any light in viewspace, dunno if that is something I am failing on somehow. Or maybe I am just crazy, certently starting to feel crazy....  
> **username_2**: The depth value was stored with the correct sign, but then reconstructed with the wrong sign, is what I'm saying. The original sign is not getting restored by the code you posted.  
> **username_2**: Re: "Move the light to a negative Z-value" never mind that, I missed that the light position is in world space not view space. In any case, after transforming by the ViewMatrix it should end up with a negative Z value since it's in front of the camera.  
> **username_0**: Okok. Hmm does not help so far =/. Light still rotating aound  
> **username_2**: BTW for debugging, try visualizing the XYZ positions as RGB, by setting your shaders to output `fract(vPos)` to see the view position from the first pass, or `fract(position)` to see the reconstructed position in the second pass. The results from those two images should match.  
> **username_0**: Much love! You dont know hos much i been trying. Will do in a moment! And post screen  
> **username_0**: Not sure how to look at that. But I guess it looks wrong? Not sure.  I added screenshots above  
> **username_2**: OK. So you've got your fract(position) from the second pass, now what you need to do is write out fract(vPos) from the first pass and compare that. The first pass vPos is the ground truth. The second pass position needs to be made to match that.  
> **username_0**: I added GIF's with that, but dont know how to go forward from here, I am running out of ideas  

---

