<p>Without looking at the code, from the look of the pictures, my intuition would be that somewhere the code is using the inverse of the matrix it should.</p>

<p>Looking at the code seems to confirm so, but there are some other problems too.</p>

<blockquote>
<pre><code>mTBN = transpose(mat3(t,b,n)); //transpose makes it from model to to tangent space?
</code></pre>
</blockquote>

<p><code>mat3(t,b,n)</code> is a transform from world space to tangent space. The inverse of such a matrix is indeed its transpose, so <code>mTBN</code> transforms from tangent space to world space, which is not what you want.</p>

<blockquote>
<pre><code>tanViewPos = mTBN * cameraPos;
tanFragPos = mTBN * worldPos.xyz;
</code></pre>
</blockquote>

<p>You cannot do something like that. <code>mTBN</code> is a 3x3 matrix: without a 4th column and a non-zero homogeneous coordinate, you can only transform directions, not positions. You need to first get the camera direction relative to the vertex, then transform that direction.</p>

<pre><code>// Beware of the vector direction.
// Here viewDir points from the object toward the viewer.
vec3 worldViewDir = normalize(cameraPos - worldPos.xyz);
tanViewDir = mTBN * worldViewDir;
</code></pre>

<p>Let us know in comments if that helps solving your issue.</p>

CC BY-SA 3.0
2016-09-30T14:11:04.027Z
I made some edit
mdkdy
CC BY-SA 3.0
2016-09-29T14:21:12.513Z
2016-09-29T14:27:04.720Z
2016-09-29T14:27:04.720Z
Julien Guertault
<p>you don't actually need to transform the z component, just ommit it and replace it with 1,because it is in tangent space it will only cause problems for this
edit: I had the same problem and that fixed it</p>

CC BY-SA 4.0
2020-05-30T03:58:53.173Z
2020-05-30T03:58:53.173Z
Eyad Ahmed
<p>I'm trying to implement POM using <a href="http://learnopengl.com/#!Advanced-Lighting/Parallax-Mapping" rel="nofollow noreferrer">this tutorial</a>. As stated, I need to transform fragment-to-camera to tangent space and then pass it to a function.</p>

<ul>
<li>tangent space is computed using <a href="http://www.assimp.org" rel="nofollow noreferrer">Assimp</a>. Well I trusted it. For example tangent vectors look like that (Sponza model):</li>
</ul>

<p><a href="https://i.sstatic.net/OdSJs.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/OdSJsm.png" alt="enter image description here"></a></p>

<ul>
<li><p>then in vertex I make transformation as in tutorial:</p>

<p><pre><code>layout (location = 0) in vec3 vPosition;
layout (location = 1) in vec2 vTexCoord;
layout (location = 2) in vec3 vNormal;
layout (location = 3) in vec3 vTangent;
layout (location = 4) in vec3 vBitangent;
...
worldPos = matrices.modelMatrix * vec4(vPosition, 1.0);
//transform t, b, n from object to world (also called model) space?
vec3 t = normalize(mat3(matrices.modelMatrix) * vTangent);
vec3 b = normalize(mat3(matrices.modelMatrix) * vBitangent);
vec3 n = normalize(mat3(matrices.modelMatrix) * vNormal);
mTBN = transpose(mat3(t,b,n)); //transpose makes it from model to to tangent space?
tanViewPos = mTBN * cameraPos;
tanFragPos = mTBN * worldPos.xyz;</pre></code></p></li>
<li><p>then in fragment I pass parameters to function:</p>

<p><pre><code>vec3 viewDir = normalize(tanViewPos - tanFragPos);
    texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);</pre></code></p>

<p><pre><code>vec2 steepParallaxMapping(vec2 texCoords, vec3 viewDir, sampler2D textureHeight)
{
float height_scale = 0.09;
// number of depth layers
const float numLayers = 30.0;
// calculate the size of each layer
float layerDepth = 1.0 / numLayers;
// depth of current layer
float currentLayerDepth = 0.0;
// the amount to shift the texture coordinates per layer (from vector P)
vec2 P = viewDir.xy * height_scale;
vec2 deltaTexCoords = P / numLayers;
// get initial values
vec2  currentTexCoords     = texCoords;
float currentDepthMapValue = texture2D(textureHeight, currentTexCoords).r;</p>

<p>while(currentLayerDepth &lt; currentDepthMapValue)
{
    // shift texture coordinates along direction of P
    currentTexCoords -= deltaTexCoords;
    // get depthmap value at current texture coordinates
    currentDepthMapValue = texture2D(textureHeight, currentTexCoords).r;
    // get depth of next layer
    currentLayerDepth += layerDepth;
}</p>

<p>// get texture coordinates before collision (reverse operations)
vec2 prevTexCoords = currentTexCoords + deltaTexCoords;</p>

<p>// get depth after and before collision for linear interpolation
float afterDepth  = currentDepthMapValue - currentLayerDepth;
float beforeDepth = texture2D(textureHeight, prevTexCoords).r - currentLayerDepth + layerDepth;</p>

<p>// interpolation of texture coordinates
float weight = afterDepth / (afterDepth - beforeDepth);
return prevTexCoords * weight + currentTexCoords * (1.0 - weight);
}</pre></code></p>

<p>Thats the result (simple displacement):</p></li>
</ul>

<p><a href="https://i.sstatic.net/XuvYL.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/XuvYLm.jpg" alt="enter image description here"></a></p>

<p>Or other texture (complex):</p>

<p><a href="https://i.sstatic.net/bI99O.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/bI99Om.jpg" alt="enter image description here"></a></p>

<p>I'm doing something wrong. And I wonder why we multiply by 3x3 part of the model matrix?</p>

<p><strong>EDIT:</strong></p>

<p>I've found function which calculates tbn in fragment:</p>

<pre><code>mat3 compute_tangent_frame(vec3 N, vec3 P, vec2 UV)
{
    // get edge vectors of the pixel triangle
    vec3 dp1 = dFdx( P );
    vec3 dp2 = dFdy( P );
    vec2 duv1 = dFdx( UV );
    vec2 duv2 = dFdy( UV );

    // solve the linear system
    vec3 dp2perp = cross( dp2, N );
    vec3 dp1perp = cross( N, dp1 );
    vec3 T = dp2perp * duv1.x + dp1perp * duv2.x;
    vec3 B = dp2perp * duv1.y + dp1perp * duv2.y;

    // construct a scale-invariant frame
    float invmax = inversesqrt( max( dot(T,T), dot(B,B) ) );
    return mat3( T * invmax, B * invmax, N );
}
</code></pre>

<p><strong>First Case:</strong> So if I use that function like that:</p>

<pre><code>//in vertex
vFar = cameraPos - worldPos.xyz; 
//in fragment
mat3 TBN = (compute_tangent_frame(vNormal0, normalize(vFar), vTexCoord0));
vec3 viewDir = (transpose(TBN)*normalize(cameraPos - worldPos.xyz));
vec2 texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);
vNormalExtr = normalize(texture2D(textureNormal, texCoord).rgb*2.0 - 1.0);
normalMapColor = vec4(normalize((TBN) * vNormalExtr), 1.0);
</code></pre>

<p>parallax (all walls) and normal light seem to be ok:</p>

<p><a href="https://i.sstatic.net/1YKhy.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1YKhym.jpg" alt="enter image description here"></a></p>

<p><strong>Second Case:</strong> But if I use my TBN from CPU:</p>

<pre><code>//in vertex
mat4 model = ((matrices.modelMatrix));
vec3 t = normalize(mat3(model) * (vTangent));
vec3 b = normalize(mat3(model) * (vBitangent));
vec3 n = normalize(mat3(model) * (vNormal));
mTBN = ((mat3(t,b,n)));
//in fragment
mat3 TBN = mTBN;
vec3 viewDir = (transpose(TBN)*normalize(cameraPos - worldPos.xyz));
vec2 texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);
vNormalExtr = normalize(texture2D(textureNormal, texCoord).rgb*2.0 - 1.0);
normalMapColor = vec4(normalize((TBN) * vNormalExtr), 1.0);
</code></pre>

<p>parallax is still twisted on all walls but normal light seem to be ok:</p>

<p><a href="https://i.sstatic.net/KQFIr.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/KQFIrm.jpg" alt="enter image description here"></a></p>

<p>If I dont transpose tbn (viewDir) (the correct way You said) in second case (mTBN)</p>

<pre><code>vec3 viewDir = ((TBN)*normalize(cameraPos - worldPos.xyz));
</code></pre>

<p>then some walls seem to be correct, others not...</p>

<p><strong>EDIT:</strong></p>

<p>Adventures in tangent space...
Is there a badge of longest question ever or biggest number of pictures?</p>

<p><a href="https://i.sstatic.net/2vdi7.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/2vdi7.jpg" alt="enter image description here"></a></p>

<p><a href="https://i.sstatic.net/vA5lc.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/vA5lc.png" alt="enter image description here"></a></p>

CC BY-SA 3.0
2016-10-01T02:39:50.120Z
What are the walls that don't seem correct? How do they look? How do their N and T look?
Julien Guertault
<shader><texture><transformations><glsl>
Transforming fragment-to-camera to tangent space for parallax occlusion mapping
mdkdy
# Transforming fragment-to-camera to tangent space for parallax occlusion mapping

(Asked by: username_0 on 2016-09-19)

I'm trying to implement POM using [this tutorial](http://learnopengl.com/#!Advanced-Lighting/Parallax-Mapping). As stated, I need to transform fragment-to-camera to tangent space and then pass it to a function.

* tangent space is computed using [Assimp](http://www.assimp.org). Well I trusted it. For example tangent vectors look like that (Sponza model):

[![enter image description here](https://i.sstatic.net/OdSJsm.png)](https://i.sstatic.net/OdSJs.jpg)

* then in vertex I make transformation as in tutorial:

  ```
  layout (location = 0) in vec3 vPosition;
  layout (location = 1) in vec2 vTexCoord;
  layout (location = 2) in vec3 vNormal;
  layout (location = 3) in vec3 vTangent;
  layout (location = 4) in vec3 vBitangent;
  ...
  worldPos = matrices.modelMatrix * vec4(vPosition, 1.0);
  //transform t, b, n from object to world (also called model) space?
  vec3 t = normalize(mat3(matrices.modelMatrix) * vTangent);
  vec3 b = normalize(mat3(matrices.modelMatrix) * vBitangent);
  vec3 n = normalize(mat3(matrices.modelMatrix) * vNormal);
  mTBN = transpose(mat3(t,b,n)); //transpose makes it from model to to tangent space?
  tanViewPos = mTBN * cameraPos;
  tanFragPos = mTBN * worldPos.xyz;
  ```
* then in fragment I pass parameters to function:

  ```
  vec3 viewDir = normalize(tanViewPos - tanFragPos);
      texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);
  ```

  ```
  vec2 steepParallaxMapping(vec2 texCoords, vec3 viewDir, sampler2D textureHeight)
  {
  float height_scale = 0.09;
  // number of depth layers
  const float numLayers = 30.0;
  // calculate the size of each layer
  float layerDepth = 1.0 / numLayers;
  // depth of current layer
  float currentLayerDepth = 0.0;
  // the amount to shift the texture coordinates per layer (from vector P)
  vec2 P = viewDir.xy * height_scale;
  vec2 deltaTexCoords = P / numLayers;
  // get initial values
  vec2  currentTexCoords     = texCoords;
  float currentDepthMapValue = texture2D(textureHeight, currentTexCoords).r;

  while(currentLayerDepth < currentDepthMapValue)
  {
      // shift texture coordinates along direction of P
      currentTexCoords -= deltaTexCoords;
      // get depthmap value at current texture coordinates
      currentDepthMapValue = texture2D(textureHeight, currentTexCoords).r;
      // get depth of next layer
      currentLayerDepth += layerDepth;
  }

  // get texture coordinates before collision (reverse operations)
  vec2 prevTexCoords = currentTexCoords + deltaTexCoords;

  // get depth after and before collision for linear interpolation
  float afterDepth  = currentDepthMapValue - currentLayerDepth;
  float beforeDepth = texture2D(textureHeight, prevTexCoords).r - currentLayerDepth + layerDepth;

  // interpolation of texture coordinates
  float weight = afterDepth / (afterDepth - beforeDepth);
  return prevTexCoords * weight + currentTexCoords * (1.0 - weight);
  }

  ```

  Thats the result (simple displacement):

[![enter image description here](https://i.sstatic.net/XuvYLm.jpg)](https://i.sstatic.net/XuvYL.jpg)

Or other texture (complex):

[![enter image description here](https://i.sstatic.net/bI99Om.jpg)](https://i.sstatic.net/bI99O.jpg)

I'm doing something wrong. And I wonder why we multiply by 3x3 part of the model matrix?

**EDIT:**

I've found function which calculates tbn in fragment:

```
mat3 compute_tangent_frame(vec3 N, vec3 P, vec2 UV)
{
    // get edge vectors of the pixel triangle
    vec3 dp1 = dFdx( P );
    vec3 dp2 = dFdy( P );
    vec2 duv1 = dFdx( UV );
    vec2 duv2 = dFdy( UV );

    // solve the linear system
    vec3 dp2perp = cross( dp2, N );
    vec3 dp1perp = cross( N, dp1 );
    vec3 T = dp2perp * duv1.x + dp1perp * duv2.x;
    vec3 B = dp2perp * duv1.y + dp1perp * duv2.y;

    // construct a scale-invariant frame
    float invmax = inversesqrt( max( dot(T,T), dot(B,B) ) );
    return mat3( T * invmax, B * invmax, N );
}

```

**First Case:** So if I use that function like that:

```
//in vertex
vFar = cameraPos - worldPos.xyz; 
//in fragment
mat3 TBN = (compute_tangent_frame(vNormal0, normalize(vFar), vTexCoord0));
vec3 viewDir = (transpose(TBN)*normalize(cameraPos - worldPos.xyz));
vec2 texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);
vNormalExtr = normalize(texture2D(textureNormal, texCoord).rgb*2.0 - 1.0);
normalMapColor = vec4(normalize((TBN) * vNormalExtr), 1.0);

```

parallax (all walls) and normal light seem to be ok:

[![enter image description here](https://i.sstatic.net/1YKhym.jpg)](https://i.sstatic.net/1YKhy.jpg)

**Second Case:** But if I use my TBN from CPU:

```
//in vertex
mat4 model = ((matrices.modelMatrix));
vec3 t = normalize(mat3(model) * (vTangent));
vec3 b = normalize(mat3(model) * (vBitangent));
vec3 n = normalize(mat3(model) * (vNormal));
mTBN = ((mat3(t,b,n)));
//in fragment
mat3 TBN = mTBN;
vec3 viewDir = (transpose(TBN)*normalize(cameraPos - worldPos.xyz));
vec2 texCoord = steepParallaxMapping(vTexCoord0, viewDir, textureHeight);
vNormalExtr = normalize(texture2D(textureNormal, texCoord).rgb*2.0 - 1.0);
normalMapColor = vec4(normalize((TBN) * vNormalExtr), 1.0);

```

parallax is still twisted on all walls but normal light seem to be ok:

[![enter image description here](https://i.sstatic.net/KQFIrm.jpg)](https://i.sstatic.net/KQFIr.jpg)

If I dont transpose tbn (viewDir) (the correct way You said) in second case (mTBN)

```
vec3 viewDir = ((TBN)*normalize(cameraPos - worldPos.xyz));

```

then some walls seem to be correct, others not...

**EDIT:**

Adventures in tangent space...
Is there a badge of longest question ever or biggest number of pictures?

[![enter image description here](https://i.sstatic.net/2vdi7.jpg)](https://i.sstatic.net/2vdi7.jpg)

[![enter image description here](https://i.sstatic.net/vA5lc.png)](https://i.sstatic.net/vA5lc.png)

**Tags:** shader, texture, transformations, glsl

> **Comments:**
> **username_1**: What are the walls that don't seem correct? How do they look? How do their N and T look?  

---

## 2 answers

### Answer by username_1 on 2016-09-29. Score: 4

Without looking at the code, from the look of the pictures, my intuition would be that somewhere the code is using the inverse of the matrix it should.

Looking at the code seems to confirm so, but there are some other problems too.

> ```
> mTBN = transpose(mat3(t,b,n)); //transpose makes it from model to to tangent space?
>
> ```

`mat3(t,b,n)` is a transform from world space to tangent space. The inverse of such a matrix is indeed its transpose, so `mTBN` transforms from tangent space to world space, which is not what you want.

> ```
> tanViewPos = mTBN * cameraPos;
> tanFragPos = mTBN * worldPos.xyz;
>
> ```

You cannot do something like that. `mTBN` is a 3x3 matrix: without a 4th column and a non-zero homogeneous coordinate, you can only transform directions, not positions. You need to first get the camera direction relative to the vertex, then transform that direction.

```
// Beware of the vector direction.
// Here viewDir points from the object toward the viewer.
vec3 worldViewDir = normalize(cameraPos - worldPos.xyz);
tanViewDir = mTBN * worldViewDir;

```

Let us know in comments if that helps solving your issue.

> **Comments:**
> **username_0**: I made some edit  

---

### Answer by username_2 on 2020-05-30. Score: 0

you don't actually need to transform the z component, just ommit it and replace it with 1,because it is in tangent space it will only cause problems for this
edit: I had the same problem and that fixed it


---

