<p>Mip selection is pretty well standardized across devices today—with the exception of some of the nitty-gritty details of anisotropic filtering, which is still up to the individual GPU manufacturers to define (and its precise details are generally not publicly documented).</p>
<p>A good place to read about mip selection  in detail is in <a href="https://registry.khronos.org/OpenGL/specs/gl/glspec46.core.pdf#section.8.14" rel="nofollow noreferrer">the OpenGL spec, section 8.14, &quot;Texture Minification&quot;</a>. I'd assume it works the same way in Metal. (Apple <em>could</em> have changed something, considering they make both the hardware and the API...but I doubt they have.) I'll summarize it here.</p>
<p>The default mip selection (not using any of the <code>lod_options</code> modifiers) uses the screen-space gradients of the texture coordinates to pick the mip levels. Essentially, it tries to pick the mip levels that produce as close as possible to a 1:1 mapping of texels to pixels. For example, if the gradients have a length of 4 texels per pixel, it would pick mip level 2 (which is 1/4th the size of level 0 and therefore gives you 1 mipped texel per pixel).</p>
<p>With trilinear filtering, since you usually don't land on an <em>exact</em> 1:1 mapping, it picks the two nearest levels and linearly interpolates between them, so that you have a smooth transition between mip levels as the camera or objects in your scene move around.</p>
<p>To be mathematically precise: you take the screen-space gradients of the texture coordinates, scale them by the texture size (to get them in units of &quot;texels per pixel&quot;), take their lengths, take the longer of the two (X and Y) gradients, and calculate its logarithm base 2. This quantity is called <span class="math-container">$\lambda$</span> in the OpenGL spec; it's also commonly known as the texture LOD (level of detail). It's simply a continuous version of the integer mip level index. So, the mip levels used are the two integers nearest to <span class="math-container">$\lambda$</span>, and the fractional part of <span class="math-container">$\lambda$</span> is used to blend between them. For example if <span class="math-container">$\lambda = 2.8$</span>, the GPU will sample mip levels 2 and 3 of the texture, then blend to 80% level 3 and 20% level 2.</p>
<p>If anisotropic filtering is turned on, then instead of simply using the longer of the two gradients, you use the ratio of them to set the number of aniso samples. For example, if the X gradient is 4 times longer than the Y gradient, you'd use 4 aniso samples, with their positions spaced out along the X gradient. Each one would be a trilinear sample, using a <span class="math-container">$\lambda$</span> corresponding to 1/4th the length of the X gradient (i.e. two mip levels lower, since <span class="math-container">$\log_2(1/4) = -2$</span>).</p>
<p>Now, as for the modifier options:</p>
<ul>
<li><code>bias</code> applies an offset to <span class="math-container">$\lambda$</span> before using it to select mip levels. So, for example, a bias of +1 will make it do trilinear filtering using mips one level higher than it would normally. A bias of −1 will use mips one level lower, and so on.</li>
<li><code>level</code> completely overrides the automatic calculation of <span class="math-container">$\lambda$</span> from screen-space gradients, and lets you put in your own value of <span class="math-container">$\lambda$</span> (aka <code>lod</code>) directly.</li>
<li><code>gradient2d</code> lets you put in your own gradient vectors, which substitute for the implicit screen-space gradients of the texture coordinates. The rest of the mip selection and sampling process proceeds as normal, but with the altered gradients. This lets you customize anisotropic filtering.</li>
</ul>
<p>In other types of shaders besides fragment shaders, there's no notion of &quot;screen-space gradients&quot;, so the latter two operations are typically the only ones allowed—any operation that tries to use implicit gradients would give a compile error. I'm not positive that's how Metal does it, but that's what I'd expect from working with other APIs.</p>

CC BY-SA 4.0
2017-09-02T23:29:48.480Z
2022-08-19T18:19:11.687Z
2022-08-19T18:19:11.687Z
Nathan Reed
<p>My question is specifically in regards to Metal, since I don't know if the answer would change for another API.</p>

<p>What I believe I understand so far is this:</p>

<ul>
<li><p>A mipmapped texture has precomputed "levels of detail", where lower levels of detail are created by downsampling the original texture in some meaningful way.</p></li>
<li><p>Mipmap levels are referred to in descending level of detail, where level <code>0</code> is the original texture, and higher-levels are power-of-two reductions of it.</p></li>
<li><p>Most GPUs implement trilinear filtering, which picks two neighboring mipmap levels for each sample, samples from each level using bilinear filtering, and then linearly blends those samples.</p></li>
</ul>

<p>What I don't quite understand is how these mipmap levels are selected. In the <a href="https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf" rel="noreferrer">documentation</a> for the Metal standard library, I see that samples can be taken, with or without specifying an instance of a <code>lod_options</code> type. I would assume that this argument changes how the mipmap levels are selected, and there are apparently three kinds of <code>lod_options</code> for 2D textures:</p>

<ul>
<li><code>bias(float value)</code></li>
<li><code>level(float lod)</code></li>
<li><code>gradient2d(float2 dPdx, float2 dPdy)</code></li>
</ul>

<p>Unfortunately, the documentation doesn't bother explaining what any of these options do. I can guess that <code>bias()</code> biases some automatically chosen level of detail, but then what does the bias <code>value</code> mean? What scale does it operate on? Similarly, how is the <code>lod</code> of <code>level()</code> translated into discrete mipmap levels? And, operating under the assumption that <code>gradient2d()</code> uses the gradient of the texture coordinate, how does it use that gradient to select the mipmap level?</p>

<p>More importantly, if I omit the <code>lod_options</code>, how are the mipmap levels selected then? Does this differ depending on the type of function being executed?</p>

<p>And, if the default no-lod-options-specified operation of the <code>sample()</code> function is to do something like <code>gradient2D()</code> (at least in a fragment shader), does it utilize simple screen-space derivatives, or does it work directly with rasterizer and interpolated texture coordinates to calculate a precise gradient?</p>

<p>And finally, how consistent is any of this behavior from device to device? An old article (old as in DirectX 9) I read referred to complex device-specific mipmap selection, but I don't know if mipmap selection is better-defined on newer architectures.</p>

<texture><interpolation><rasterizer><filtering><mipmap>
How are mipmap levels computed in Metal?
lcmylin
# How are mipmap levels computed in Metal?

(Asked by: username_0 on 2017-09-01)

My question is specifically in regards to Metal, since I don't know if the answer would change for another API.

What I believe I understand so far is this:

* A mipmapped texture has precomputed "levels of detail", where lower levels of detail are created by downsampling the original texture in some meaningful way.
* Mipmap levels are referred to in descending level of detail, where level `0` is the original texture, and higher-levels are power-of-two reductions of it.
* Most GPUs implement trilinear filtering, which picks two neighboring mipmap levels for each sample, samples from each level using bilinear filtering, and then linearly blends those samples.

What I don't quite understand is how these mipmap levels are selected. In the [documentation](https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf) for the Metal standard library, I see that samples can be taken, with or without specifying an instance of a `lod_options` type. I would assume that this argument changes how the mipmap levels are selected, and there are apparently three kinds of `lod_options` for 2D textures:

* `bias(float value)`
* `level(float lod)`
* `gradient2d(float2 dPdx, float2 dPdy)`

Unfortunately, the documentation doesn't bother explaining what any of these options do. I can guess that `bias()` biases some automatically chosen level of detail, but then what does the bias `value` mean? What scale does it operate on? Similarly, how is the `lod` of `level()` translated into discrete mipmap levels? And, operating under the assumption that `gradient2d()` uses the gradient of the texture coordinate, how does it use that gradient to select the mipmap level?

More importantly, if I omit the `lod_options`, how are the mipmap levels selected then? Does this differ depending on the type of function being executed?

And, if the default no-lod-options-specified operation of the `sample()` function is to do something like `gradient2D()` (at least in a fragment shader), does it utilize simple screen-space derivatives, or does it work directly with rasterizer and interpolated texture coordinates to calculate a precise gradient?

And finally, how consistent is any of this behavior from device to device? An old article (old as in DirectX 9) I read referred to complex device-specific mipmap selection, but I don't know if mipmap selection is better-defined on newer architectures.

**Tags:** texture, interpolation, rasterizer, filtering, mipmap


---

## 1 answers

### Answer by username_1 on 2017-09-02. Score: 23 | Accepted answer

Mip selection is pretty well standardized across devices today—with the exception of some of the nitty-gritty details of anisotropic filtering, which is still up to the individual GPU manufacturers to define (and its precise details are generally not publicly documented).

A good place to read about mip selection in detail is in [the OpenGL spec, section 8.14, "Texture Minification"](https://registry.khronos.org/OpenGL/specs/gl/glspec46.core.pdf#section.8.14). I'd assume it works the same way in Metal. (Apple *could* have changed something, considering they make both the hardware and the API...but I doubt they have.) I'll summarize it here.

The default mip selection (not using any of the `lod_options` modifiers) uses the screen-space gradients of the texture coordinates to pick the mip levels. Essentially, it tries to pick the mip levels that produce as close as possible to a 1:1 mapping of texels to pixels. For example, if the gradients have a length of 4 texels per pixel, it would pick mip level 2 (which is 1/4th the size of level 0 and therefore gives you 1 mipped texel per pixel).

With trilinear filtering, since you usually don't land on an *exact* 1:1 mapping, it picks the two nearest levels and linearly interpolates between them, so that you have a smooth transition between mip levels as the camera or objects in your scene move around.

To be mathematically precise: you take the screen-space gradients of the texture coordinates, scale them by the texture size (to get them in units of "texels per pixel"), take their lengths, take the longer of the two (X and Y) gradients, and calculate its logarithm base 2. This quantity is called $\lambda$ in the OpenGL spec; it's also commonly known as the texture LOD (level of detail). It's simply a continuous version of the integer mip level index. So, the mip levels used are the two integers nearest to $\lambda$, and the fractional part of $\lambda$ is used to blend between them. For example if $\lambda = 2.8$, the GPU will sample mip levels 2 and 3 of the texture, then blend to 80% level 3 and 20% level 2.

If anisotropic filtering is turned on, then instead of simply using the longer of the two gradients, you use the ratio of them to set the number of aniso samples. For example, if the X gradient is 4 times longer than the Y gradient, you'd use 4 aniso samples, with their positions spaced out along the X gradient. Each one would be a trilinear sample, using a $\lambda$ corresponding to 1/4th the length of the X gradient (i.e. two mip levels lower, since $\log\_2(1/4) = -2$).

Now, as for the modifier options:

* `bias` applies an offset to $\lambda$ before using it to select mip levels. So, for example, a bias of +1 will make it do trilinear filtering using mips one level higher than it would normally. A bias of −1 will use mips one level lower, and so on.
* `level` completely overrides the automatic calculation of $\lambda$ from screen-space gradients, and lets you put in your own value of $\lambda$ (aka `lod`) directly.
* `gradient2d` lets you put in your own gradient vectors, which substitute for the implicit screen-space gradients of the texture coordinates. The rest of the mip selection and sampling process proceeds as normal, but with the altered gradients. This lets you customize anisotropic filtering.

In other types of shaders besides fragment shaders, there's no notion of "screen-space gradients", so the latter two operations are typically the only ones allowed—any operation that tries to use implicit gradients would give a compile error. I'm not positive that's how Metal does it, but that's what I'd expect from working with other APIs.


---

