<p>Usually the tangent and normal for a given vertex TBN is passed in as a vertex attribute, something along the lines of this...</p>
<pre><code>x,y,z //vertex position
x,y,z //normal
x,y,z,w //tangent where &quot;w&quot; represents the direction of the bitangent
u,v //texture coordinates
</code></pre>
<p>Given that set of attributes:
The bitangent is computed by taking the cross product of vertex attributes tangent and normal, then multiplied by the w coordinated of the tangent to get the correct direction. The w component is either 1 or -1. This gives us our TBN.</p>
<p>The fragment shader uses the UV coordinates to lookup the associated normal from a normal map. Then uses the TBN computed above to compute a normal that will be used for lighting.</p>
<p>That's pretty much it.</p>
<p>All the hard work is precomputed. A normal map is computed from the height map. The TBN for the mesh vertex attributes is also precomputed.</p>
<p>One way to compute the TBN is using the UV coordinates and vertex positions to create a system of equations which can be solved to find the tangent and bitangent.</p>
<p>The TBN values are also typically &quot;averaged&quot; together with neighboring TBN vectors, and finally orthonormalized to get a better tangent frame.</p>
<hr />
<p>When computing the Tangent frame, <em>u</em> and <em>v</em> coordinates for the vertex attributes give distances along the <em>t</em> and <em>b</em> axes and are aligned to the <em>x</em> and <em>y</em> directions of the texture map.</p>
<p>This means that the <em>T</em> and <em>B</em> vectors of the TBN rotate vectors from the normal map in the <em>x</em>, <em>y</em> plane to align with triangles in the mesh.</p>
<p>If we then take that tangent frame and attempt to use it with vectors derived from derivatives of values in a height map, we may end up with <em>T</em> and <em>B</em> vectors that are in the correct plane, but those vectors will be rotated so they point in directions aligned with texture space of the normal map they are created for. This will result in transformed normals that are correct when they align with the original texture map, but will otherwise point in directions that give unusable results.</p>
<hr />
<p>Another approach to this problem that is somewhat similar to what you are describing is called &quot;derivative maps&quot;. I recommend looking up that technique.</p>

CC BY-SA 4.0
2022-05-26T18:40:08.920Z
Yes, I have a TBN like this. In my code example, it is coming from vertex attributes. I want to use it with a height map though, not with a pre-made Normal map. The goal is bump mapping with arbitrary height input, but using precomputed Tangents instead of position computed tangents, since those are lower quality.
Ascalon
CC BY-SA 4.0
2022-05-30T00:30:49.923Z
Actually Mickelson continued work on the paper you are referencing and has a newer published paper that goes into good detail on the subject including the math in the journal of computer graphics including a working program. You can find it here: https://jcgt.org/published/0009/03/04/
pmw1234
CC BY-SA 4.0
2022-05-30T01:07:36.600Z
Yes, I am familiar with it. It does not address this specific issue as far as I can tell. I may be missing something though. It has the bump map from unparametrized surfaces (same as above), and it has the triple tap forward differencing for image textures (which I have working.) But does not seem to have arbitrary height input with precomputed tangents.
Ascalon
CC BY-SA 4.0
2022-05-31T23:33:35.643Z
I am also aware of derivative maps, but they don't solve this issue either. The question is about Normalization, and issues with this specific combination of Precomputed Tangent and dFdx/y derivatives. I appreciate the other info, but the question is not about alternatives. I am already aware of them.
Ascalon
CC BY-SA 4.0
2022-05-31T23:51:14.857Z
Right, but the precomputed tangents that are computed with UV coords in texture space are not in the same space as derivate's that are computed from screen space using the chain rule and the only way to solve the problem is either translate the TBN or compute  directly from the height map both of which make the entire endeavor pointless, since  in the first place the alu cost is way to much and in the second place is how the normals are computed to begin with.
pmw1234
CC BY-SA 4.0
2022-06-01T04:11:54.997Z
I'm interested to know about translating the TBN or directly computing. I'm doing offline rendering here so I'm not concerned about the performance. Well, mostly I'm just trying to understand things relative to each other. I have working computed tangent+computed derivs. And I've got precomputed tangents + UV offset based height derivs. I want to understand if I can get precomputed tangent + computed height derivs, and at what cost, and if its impossible or just infeasible.
Ascalon
CC BY-SA 4.0
2022-05-26T11:40:36.003Z
2022-05-29T15:11:14.133Z
2022-05-29T15:11:14.133Z
pmw1234
<p>Most bump mapping (meaning black and white height map) uses Tangents/Bitangents computed from Position using dFdx/y. And they also compute the offsets/derivatives of the height map using dFdx/y.  The standard seems to be to not use precomputed Tangents/Bitangents. If you search for bump mapping, everything to do with it seems to be like this. I am trying to get a setup working that does use precomputed tangents, as these are higher quality. But I do not know how to adapt the math to work with them.</p>
<p>Here is an example of typical bump mapping based on <a href="https://mmikk.github.io/papers3d/mm_sfgrad_bump.pdf" rel="nofollow noreferrer">Mikkelsen2010</a>. (This is actually the setup that Blender uses for the <a href="https://github.com/blender/blender/blob/master/source/blender/gpu/shaders/material/gpu_shader_material_bump.glsl" rel="nofollow noreferrer">bump mapping node</a> in its Eevee render engine. I have recreated it in the <a href="https://github.com/bnpr/Malt" rel="nofollow noreferrer">Malt render engine</a>.)</p>
<pre><code>void bump_without_precomputed_tangents(
    float strength, float dist, float height, float invert, out vec3 result)
{
#ifdef PIXEL_SHADER
  vec3 N = normalize(NORMAL);
  dist *= gl_FrontFacing ? invert : -invert;

  vec3 dPdx = dFdx(POSITION);
  vec3 dPdy = dFdy(POSITION);

  //Get surface tangents from normal.
  vec3 Rx = cross(dPdy, N);
  vec3 Ry = cross(N, dPdx);

  //Calculate height derivatives (usually done in another function)
  vec2 dHd = vec2(dFdx(height), dFdy(height));

  //Compute surface gradient and determinant.
  float det = dot(dPdx, Rx);

  vec3 surfgrad = dHd.x * Rx + dHd.y * Ry;

  strength = max(strength, 0.0);

  result = normalize(abs(det) * N - dist * sign(det) * surfgrad);
  result = normalize(mix(N, result, strength));

#else
  result = (NORMAL);
#endif
}
</code></pre>
<p>Here's a test setup showing a gradient sphere height map texture:
<a href="https://i.sstatic.net/LpY4k.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/LpY4k.png" alt="enter image description here" /></a></p>
<p>And the Normal from using it with the above code (using Malt 1.0 in Blender 3.1):
<a href="https://i.sstatic.net/7kXjP.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/7kXjP.png" alt="enter image description here" /></a></p>
<p>You can see there are problems of ridge-like noise and the geometry showing in the Normal. These are normal for this style of bump mapping. The ridges are caused by using dFdx/y for the height map derivatives. The mesh showing is partially from that, and partially from using dFdx/y on Position to get the Tangent/Bitangent. If proper precomputed Tangents are used, these problems will be reduced. (The derivatives can also be improved in various ways.)</p>
<p>The problem is that the precomputed tangent/bitangent are Normalized, but in the above code, dPdx/y, Rx/y, and dHd are all very low values. If I simply replace Rx/y with the precomputed Tangent/Bitangent, it doesn't work due to this. I am trying to figure out how to get dHd Normalized without causing problems. Or, the precomputed Tangents need to be scaled into the proper range without reintroducing these problems. (The precomputed tangents are provided by the Malt render engine.)</p>
<p>Here is code where I am bringing in the precomputed tangents and Normalizing dHd. But this does not give the proper Normals in the end.</p>
<pre><code>void bump_normalized(
    float strength, float dist, float height, float invert, out vec3 result)
{
#ifdef PIXEL_SHADER
  vec3 T = get_tangent(0);
  vec3 B = get_bitangent(0);
  vec3 N = normalize(NORMAL);

  dist *= gl_FrontFacing ? invert : -invert;

  vec3 dPdx = dFdx(POSITION);
  vec3 dPdy = dFdy(POSITION);

  //Calculate height derivatives.
  vec2 dHd = vec2(dFdx(height), dFdy(height));

  //Normalize derivs so they work with Normalized Tangent. Avoid div by 0.
  dHd = length(dHd) == vec2(0.0) ? vec2(0.0) : normalize(dHd);

  //Compute surface gradient and determinant.
  float det = dot(normalize(dPdx), T);

  vec3 surfgrad = (dHd.x * T + dHd.y * B);

  strength = max(strength, 0.0);

  //result = normalize(abs(det) * N - dist * sign(det) * surfgrad);
  result = normalize(N - dist * surfgrad);
  result = normalize(mix(N, result, strength));

#else
  result = (NORMAL);
#endif
}
</code></pre>
<p>The Normal looks like this. It is incorrect.
<a href="https://i.sstatic.net/PrhTa.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/PrhTa.png" alt="enter image description here" /></a></p>
<p>In this code, I also do not know what to do with the det, as it is based on dPdx, which isn't needed now (its usual result line is commented out for now.)</p>
<p>I have other setups that successfully use the precomputed tangents and UV offset based derivatives instead, and these work fine (and are clean as no dFdx/y is involved.)</p>
<p>Can anyone tell me what I'm doing wrong here, or if there's some other way entirely to approach this? I want to be able to properly use Normalized T and B.</p>

CC BY-SA 4.0
2022-05-23T13:56:17.373Z
"*Most bump mapping (meaning black and white height map) uses Tangents/Bitangents computed from Position using dFdx/y.*" While some tutorials may do that, that's not how it's generally done in any remotely professional code.
Nicol Bolas
CC BY-SA 4.0
2022-05-23T19:46:42.383Z
@NicolBolas I am aware that this is an older way of doing things. It doesn't surprise me to hear that its not being done this way in modern professional setups. So are they using precomputed tangents then? Can you point me to any examples that may answer the question?
Ascalon
<opengl><rendering><glsl><normal-mapping>
Bump mapping WITH precomputed tangents?
Ascalon
# Bump mapping WITH precomputed tangents?

(Asked by: username_0 on 2022-05-23)

Most bump mapping (meaning black and white height map) uses Tangents/Bitangents computed from Position using dFdx/y. And they also compute the offsets/derivatives of the height map using dFdx/y. The standard seems to be to not use precomputed Tangents/Bitangents. If you search for bump mapping, everything to do with it seems to be like this. I am trying to get a setup working that does use precomputed tangents, as these are higher quality. But I do not know how to adapt the math to work with them.

Here is an example of typical bump mapping based on [Mikkelsen2010](https://mmikk.github.io/papers3d/mm_sfgrad_bump.pdf). (This is actually the setup that Blender uses for the [bump mapping node](https://github.com/blender/blender/blob/master/source/blender/gpu/shaders/material/gpu_shader_material_bump.glsl) in its Eevee render engine. I have recreated it in the [Malt render engine](https://github.com/bnpr/Malt).)

```
void bump_without_precomputed_tangents(
    float strength, float dist, float height, float invert, out vec3 result)
{
#ifdef PIXEL_SHADER
  vec3 N = normalize(NORMAL);
  dist *= gl_FrontFacing ? invert : -invert;

  vec3 dPdx = dFdx(POSITION);
  vec3 dPdy = dFdy(POSITION);

  //Get surface tangents from normal.
  vec3 Rx = cross(dPdy, N);
  vec3 Ry = cross(N, dPdx);

  //Calculate height derivatives (usually done in another function)
  vec2 dHd = vec2(dFdx(height), dFdy(height));

  //Compute surface gradient and determinant.
  float det = dot(dPdx, Rx);

  vec3 surfgrad = dHd.x * Rx + dHd.y * Ry;

  strength = max(strength, 0.0);

  result = normalize(abs(det) * N - dist * sign(det) * surfgrad);
  result = normalize(mix(N, result, strength));

#else
  result = (NORMAL);
#endif
}

```

Here's a test setup showing a gradient sphere height map texture:
[![enter image description here](https://i.sstatic.net/LpY4k.png)](https://i.sstatic.net/LpY4k.png)

And the Normal from using it with the above code (using Malt 1.0 in Blender 3.1):
[![enter image description here](https://i.sstatic.net/7kXjP.png)](https://i.sstatic.net/7kXjP.png)

You can see there are problems of ridge-like noise and the geometry showing in the Normal. These are normal for this style of bump mapping. The ridges are caused by using dFdx/y for the height map derivatives. The mesh showing is partially from that, and partially from using dFdx/y on Position to get the Tangent/Bitangent. If proper precomputed Tangents are used, these problems will be reduced. (The derivatives can also be improved in various ways.)

The problem is that the precomputed tangent/bitangent are Normalized, but in the above code, dPdx/y, Rx/y, and dHd are all very low values. If I simply replace Rx/y with the precomputed Tangent/Bitangent, it doesn't work due to this. I am trying to figure out how to get dHd Normalized without causing problems. Or, the precomputed Tangents need to be scaled into the proper range without reintroducing these problems. (The precomputed tangents are provided by the Malt render engine.)

Here is code where I am bringing in the precomputed tangents and Normalizing dHd. But this does not give the proper Normals in the end.

```
void bump_normalized(
    float strength, float dist, float height, float invert, out vec3 result)
{
#ifdef PIXEL_SHADER
  vec3 T = get_tangent(0);
  vec3 B = get_bitangent(0);
  vec3 N = normalize(NORMAL);

  dist *= gl_FrontFacing ? invert : -invert;

  vec3 dPdx = dFdx(POSITION);
  vec3 dPdy = dFdy(POSITION);

  //Calculate height derivatives.
  vec2 dHd = vec2(dFdx(height), dFdy(height));

  //Normalize derivs so they work with Normalized Tangent. Avoid div by 0.
  dHd = length(dHd) == vec2(0.0) ? vec2(0.0) : normalize(dHd);

  //Compute surface gradient and determinant.
  float det = dot(normalize(dPdx), T);

  vec3 surfgrad = (dHd.x * T + dHd.y * B);

  strength = max(strength, 0.0);

  //result = normalize(abs(det) * N - dist * sign(det) * surfgrad);
  result = normalize(N - dist * surfgrad);
  result = normalize(mix(N, result, strength));

#else
  result = (NORMAL);
#endif
}

```

The Normal looks like this. It is incorrect.
[![enter image description here](https://i.sstatic.net/PrhTa.png)](https://i.sstatic.net/PrhTa.png)

In this code, I also do not know what to do with the det, as it is based on dPdx, which isn't needed now (its usual result line is commented out for now.)

I have other setups that successfully use the precomputed tangents and UV offset based derivatives instead, and these work fine (and are clean as no dFdx/y is involved.)

Can anyone tell me what I'm doing wrong here, or if there's some other way entirely to approach this? I want to be able to properly use Normalized T and B.

**Tags:** opengl, rendering, glsl, normal-mapping

> **Comments:**
> **username_1**: "*Most bump mapping (meaning black and white height map) uses Tangents/Bitangents computed from Position using dFdx/y.*" While some tutorials may do that, that's not how it's generally done in any remotely professional code.  
> **username_0**: @username_1 I am aware that this is an older way of doing things. It doesn't surprise me to hear that its not being done this way in modern professional setups. So are they using precomputed tangents then? Can you point me to any examples that may answer the question?  

---

## 1 answers

### Answer by username_2 on 2022-05-26. Score: 1

Usually the tangent and normal for a given vertex TBN is passed in as a vertex attribute, something along the lines of this...

```
x,y,z //vertex position
x,y,z //normal
x,y,z,w //tangent where "w" represents the direction of the bitangent
u,v //texture coordinates

```

Given that set of attributes:
The bitangent is computed by taking the cross product of vertex attributes tangent and normal, then multiplied by the w coordinated of the tangent to get the correct direction. The w component is either 1 or -1. This gives us our TBN.

The fragment shader uses the UV coordinates to lookup the associated normal from a normal map. Then uses the TBN computed above to compute a normal that will be used for lighting.

That's pretty much it.

All the hard work is precomputed. A normal map is computed from the height map. The TBN for the mesh vertex attributes is also precomputed.

One way to compute the TBN is using the UV coordinates and vertex positions to create a system of equations which can be solved to find the tangent and bitangent.

The TBN values are also typically "averaged" together with neighboring TBN vectors, and finally orthonormalized to get a better tangent frame.

---

When computing the Tangent frame, *u* and *v* coordinates for the vertex attributes give distances along the *t* and *b* axes and are aligned to the *x* and *y* directions of the texture map.

This means that the *T* and *B* vectors of the TBN rotate vectors from the normal map in the *x*, *y* plane to align with triangles in the mesh.

If we then take that tangent frame and attempt to use it with vectors derived from derivatives of values in a height map, we may end up with *T* and *B* vectors that are in the correct plane, but those vectors will be rotated so they point in directions aligned with texture space of the normal map they are created for. This will result in transformed normals that are correct when they align with the original texture map, but will otherwise point in directions that give unusable results.

---

Another approach to this problem that is somewhat similar to what you are describing is called "derivative maps". I recommend looking up that technique.

> **Comments:**
> **username_0**: Yes, I have a TBN like this. In my code example, it is coming from vertex attributes. I want to use it with a height map though, not with a pre-made Normal map. The goal is bump mapping with arbitrary height input, but using precomputed Tangents instead of position computed tangents, since those are lower quality.  
> **username_2**: Actually Mickelson continued work on the paper you are referencing and has a newer published paper that goes into good detail on the subject including the math in the journal of computer graphics including a working program. You can find it here: https://jcgt.org/published/0009/03/04/  
> **username_0**: Yes, I am familiar with it. It does not address this specific issue as far as I can tell. I may be missing something though. It has the bump map from unparametrized surfaces (same as above), and it has the triple tap forward differencing for image textures (which I have working.) But does not seem to have arbitrary height input with precomputed tangents.  
> **username_0**: I am also aware of derivative maps, but they don't solve this issue either. The question is about Normalization, and issues with this specific combination of Precomputed Tangent and dFdx/y derivatives. I appreciate the other info, but the question is not about alternatives. I am already aware of them.  
> **username_2**: Right, but the precomputed tangents that are computed with UV coords in texture space are not in the same space as derivate's that are computed from screen space using the chain rule and the only way to solve the problem is either translate the TBN or compute  directly from the height map both of which make the entire endeavor pointless, since  in the first place the alu cost is way to much and in the second place is how the normals are computed to begin with.  
> **username_0**: I'm interested to know about translating the TBN or directly computing. I'm doing offline rendering here so I'm not concerned about the performance. Well, mostly I'm just trying to understand things relative to each other. I have working computed tangent+computed derivs. And I've got precomputed tangents + UV offset based height derivs. I want to understand if I can get precomputed tangent + computed height derivs, and at what cost, and if its impossible or just infeasible.  

---

