<p>The GPU hardware only supports nearest-neighbor, bilinear, trilinear, and anisotropic texture filtering. However, nothing stops you from implementing your own texture filtering in the pixel shader.</p>

<p>To do this, you'd use <a href="http://docs.gl/sl4/textureGather" rel="noreferrer"><code>textureGather</code></a> to grab a 2×2 block of texels around your sample point. Then you'd manually calculate the position of the sample point relative to the texel centers. Armed with the raw texels and the interpolation position, you can then apply whatever interpolation function you like.</p>

<p>The position calculation would be something like this:</p>

<pre><code>vec2 textureSize; // texture's width and height
vec2 offset = fract(texCoord * textureSize + (-0.5 + 1.0/512.0));
</code></pre>

<p>The 1/512 offset in there is needed as a rounding correction on certain GPUs, to ensure that <code>fract()</code>'s output steps from 1 to 0 at the exact same point that <code>textureGather</code> switches texels.</p>

<p>As an example, to re-implement bilinear filtering, you'd do this (assuming a single-channel texture for simplicity):</p>

<pre><code>vec4 texels = textureGather(...);
float result = mix(mix(texels.w, texels.z, offset.x),
                   mix(texels.x, texels.y, offset.x),
                   offset.y);
</code></pre>

<p>But in practice, you'd be replacing the above with your own interpolation logic. This will for sure be slower than using hardware bilinear interpolation, but it may be an acceptable cost for your app.</p>

<p>If you need more than 2×2 block of texels, you can also use multiple calls to <a href="http://docs.gl/sl4/textureGatherOffset" rel="noreferrer"><code>textureGatherOffset</code></a> to grab more of the surrounding region, 2×2 at a time.</p>

<hr>

<p>As for generating a higher-resolution version of the texture on the fly, that can certainly also be done. It would be an extension of texture streaming techniques that load larger mip levels from disk only when needed. Here, you'd be procedurally generating the larger mip levels instead of loading them, but it's a similar idea. Unfortunately I don't have a good reference handy for how to do this. (It's somewhat related to <a href="http://mrelusive.com/publications/presentations/2012_siggraph/Virtual_Texturing_in_Software_and_Hardware_final.pdf" rel="noreferrer">virtual texturing</a>, but less complicated—for just plain texture streaming, you don't need sparse textures or a page table, etc.)</p>

<p>However, there is unfortunately no GPU hardware support for doing the thing you suggested of avoiding the extra storage, by processing the texture only a piece at a time. Doing that in hardware (“texel shaders” that would generate texel values on-demand as they are sampled by fragment shaders) is an idea that has been proposed many times but so far has not happened.</p>

<p>It might be possible to do it in software by having fragment shaders append sample requests into a queue, then switching to a different shader to service those requests, then re-running the fragment shaders with the results...but that would take a lot of work and might not be very fast. It would for sure be a research project. </p>

<p>Unless you have a good way of determining in advance which texels are likely to be needed, it would be a lot more practical to just generate the entire texture, assuming memory and performance budgets allow for it.</p>

CC BY-SA 3.0
2018-03-28T17:37:07.523Z
Thanks, a very useful answer. As for the performance of doing the interpolation in SW rather than hardware, I'm not *too* concerned, as it's only the very close up textures which would be affected, and because I would do any complex calculations offline and bake them into a map.
barneypitt
CC BY-SA 3.0
2018-03-28T17:14:12.423Z
2018-03-28T17:14:12.423Z
Nathan Reed
<p>I suspect the answer is no, but thought I'd ask anyway.</p>

<p>It seems that other than using nearest neighbour and bilinear interpolations, there is nothing you can do to determine how two adjacent pixels are interpolated ... such as applying an edge preserving interpolation, or adding some procedural variation. </p>

<p>I'd love to be wrong! If indeed there is no way to do this, might there be a practical solution which involves creating a higher resolution (maybe much higher) texture (like a "mipmap level -2") on the fly?</p>

<p>If done naively I guess it would be very memory-hungry ... perhaps there is some way it could be done to a reusable buffer, or some way ensuring only part of the image would be processed at a time (like an OpenCL/CUDA/Compute Shader workgoup) thus limiting the buffer size. (Perhaps Vulkan could help somehow?)</p>

<p>Obviously it would be done on a LOD basis ... no need to do it on far away textures.</p>

<p>Thanks  </p>

CC BY-SA 3.0
2018-03-28T15:04:03.203Z
When you say "sub-texel shading" are you really asking "can I do texture super-sampling"? i.e. sampling the texture at a rate higher than once per screen pixel?

FWIW, anisotropic filtering is generally implemented in this sort of fashion whereby, depending on the ratio of anisotropy, additional texture samples are taken and then blended together.

Alternatively, are you asking if super-sample antialiasing is available in HW?
Simon F
CC BY-SA 3.0
2018-03-28T15:29:33.650Z
It sounds to me like you're actually trying to make your own texture sampler, so you can use other sampling techniques. Is that right?
Dan Hulme
CC BY-SA 3.0
2018-03-28T17:08:57.597Z
Dan, essentially yes. Though the interpolation would potentially use more data than just the texture being sampled (pre-computed edge map, maybe another high res texture overlayed).
barneypitt
CC BY-SA 3.0
2018-03-28T17:19:18.077Z
Simon, no, I guess I'm asking if super-sampling is available - overridable - in gpu SW (i.e. fragment shader - or compute shader if the compute shader could be fitted seamlessly into the pipeline).
barneypitt
CC BY-SA 3.0
2018-03-28T17:26:04.560Z
FWIW ... in Half Life 2, I noticed what appeared to be something like this, used everywhere. When you get close to a texture, the base texture looks very blurry, but a much higher resolution texture is overlaid. At least, this is certainly what it looks like is happening to me. Guess it's one reason why the 14 year old game still looks kind of crisp today!
barneypitt
CC BY-SA 3.0
2018-03-30T01:47:09.960Z
Not sure if its exactly what you're looking for, but Quilez has an interesting technique of separating the tex coord into int and fractional parts, and manipulating the fractional part yourself before you sample the texture. http://www.iquilezles.org/www/articles/texture/texture.htm
russ
CC BY-SA 3.0
2018-04-04T08:49:46.880Z
Russ, it's not exactly what I was looking for, but it's really interesting! I'm thinking that the underlying technique might possibly be of use in what I'm trying to do, with some modification, so thanks.
barneypitt
<shader><glsl>
Is sub-texel shading possible?
barneypitt
# Is sub-texel shading possible?

(Asked by: username_0 on 2018-03-28)

I suspect the answer is no, but thought I'd ask anyway.

It seems that other than using nearest neighbour and bilinear interpolations, there is nothing you can do to determine how two adjacent pixels are interpolated ... such as applying an edge preserving interpolation, or adding some procedural variation.

I'd love to be wrong! If indeed there is no way to do this, might there be a practical solution which involves creating a higher resolution (maybe much higher) texture (like a "mipmap level -2") on the fly?

If done naively I guess it would be very memory-hungry ... perhaps there is some way it could be done to a reusable buffer, or some way ensuring only part of the image would be processed at a time (like an OpenCL/CUDA/Compute Shader workgoup) thus limiting the buffer size. (Perhaps Vulkan could help somehow?)

Obviously it would be done on a LOD basis ... no need to do it on far away textures.

Thanks

**Tags:** shader, glsl

> **Comments:**
> **username_1**: When you say "sub-texel shading" are you really asking "can I do texture super-sampling"? i.e. sampling the texture at a rate higher than once per screen pixel?  FWIW, anisotropic filtering is generally implemented in this sort of fashion whereby, depending on the ratio of anisotropy, additional texture samples are taken and then blended together.  Alternatively, are you asking if super-sample antialiasing is available in HW?  
> **username_2**: It sounds to me like you're actually trying to make your own texture sampler, so you can use other sampling techniques. Is that right?  
> **username_0**: Dan, essentially yes. Though the interpolation would potentially use more data than just the texture being sampled (pre-computed edge map, maybe another high res texture overlayed).  
> **username_0**: username_1, no, I guess I'm asking if super-sampling is available - overridable - in gpu SW (i.e. fragment shader - or compute shader if the compute shader could be fitted seamlessly into the pipeline).  
> **username_0**: FWIW ... in Half Life 2, I noticed what appeared to be something like this, used everywhere. When you get close to a texture, the base texture looks very blurry, but a much higher resolution texture is overlaid. At least, this is certainly what it looks like is happening to me. Guess it's one reason why the 14 year old game still looks kind of crisp today!  
> **username_3**: Not sure if its exactly what you're looking for, but Quilez has an interesting technique of separating the tex coord into int and fractional parts, and manipulating the fractional part yourself before you sample the texture. http://www.iquilezles.org/www/articles/texture/texture.htm  
> **username_0**: username_3, it's not exactly what I was looking for, but it's really interesting! I'm thinking that the underlying technique might possibly be of use in what I'm trying to do, with some modification, so thanks.  

---

## 1 answers

### Answer by username_4 on 2018-03-28. Score: 5

The GPU hardware only supports nearest-neighbor, bilinear, trilinear, and anisotropic texture filtering. However, nothing stops you from implementing your own texture filtering in the pixel shader.

To do this, you'd use [`textureGather`](http://docs.gl/sl4/textureGather) to grab a 2×2 block of texels around your sample point. Then you'd manually calculate the position of the sample point relative to the texel centers. Armed with the raw texels and the interpolation position, you can then apply whatever interpolation function you like.

The position calculation would be something like this:

```
vec2 textureSize; // texture's width and height
vec2 offset = fract(texCoord * textureSize + (-0.5 + 1.0/512.0));

```

The 1/512 offset in there is needed as a rounding correction on certain GPUs, to ensure that `fract()`'s output steps from 1 to 0 at the exact same point that `textureGather` switches texels.

As an example, to re-implement bilinear filtering, you'd do this (assuming a single-channel texture for simplicity):

```
vec4 texels = textureGather(...);
float result = mix(mix(texels.w, texels.z, offset.x),
                   mix(texels.x, texels.y, offset.x),
                   offset.y);

```

But in practice, you'd be replacing the above with your own interpolation logic. This will for sure be slower than using hardware bilinear interpolation, but it may be an acceptable cost for your app.

If you need more than 2×2 block of texels, you can also use multiple calls to [`textureGatherOffset`](http://docs.gl/sl4/textureGatherOffset) to grab more of the surrounding region, 2×2 at a time.

---

As for generating a higher-resolution version of the texture on the fly, that can certainly also be done. It would be an extension of texture streaming techniques that load larger mip levels from disk only when needed. Here, you'd be procedurally generating the larger mip levels instead of loading them, but it's a similar idea. Unfortunately I don't have a good reference handy for how to do this. (It's somewhat related to [virtual texturing](http://mrelusive.com/publications/presentations/2012_siggraph/Virtual_Texturing_in_Software_and_Hardware_final.pdf), but less complicated—for just plain texture streaming, you don't need sparse textures or a page table, etc.)

However, there is unfortunately no GPU hardware support for doing the thing you suggested of avoiding the extra storage, by processing the texture only a piece at a time. Doing that in hardware (“texel shaders” that would generate texel values on-demand as they are sampled by fragment shaders) is an idea that has been proposed many times but so far has not happened.

It might be possible to do it in software by having fragment shaders append sample requests into a queue, then switching to a different shader to service those requests, then re-running the fragment shaders with the results...but that would take a lot of work and might not be very fast. It would for sure be a research project.

Unless you have a good way of determining in advance which texels are likely to be needed, it would be a lot more practical to just generate the entire texture, assuming memory and performance budgets allow for it.

> **Comments:**
> **username_0**: Thanks, a very useful answer. As for the performance of doing the interpolation in SW rather than hardware, I'm not *too* concerned, as it's only the very close up textures which would be affected, and because I would do any complex calculations offline and bake them into a map.  

---

