<p>At the top level, a GPU is subdivided into a number of shader cores.  A small GPU in a notebook or tablet may have only a few cores while a high-end desktop GPU may have dozens.</p>

<p>In addition to the shader cores there are also texture units. They may be grouped together with one texture unit per shader core, or one texture unit shared among two or three shader cores, depending on the GPU.</p>

<p>The whole chip shares a single L2 cache, but the different units will have individual L1 caches.  Texture units have texture caches, and shader units have caches for instructions and constants/uniforms, and maybe a separate cache for buffer data depending on whether buffer loads are a separate path from texture loads or not (varies by GPU architecture).</p>

<p>Texture units operate independently and asynchronously from shader cores. When a shader performs a texture read, it sends a request to the texture unit across a little bus between them; the shader can then continue executing if possible, or it may get suspended and allow other shader threads to run while it waits for the texture read to finish.</p>

<p>The texture unit batches up a bunch of requests and performs the addressing math on them&mdash;selecting mip levels and anisotropy, converting UVs to texel coordinates, applying clamp/wrap modes, etc. Once it knows which texels it needs, it reads them through the cache hierarchy, the same way that memory reads work on a CPU (look in L1 first, if not there then L2, then DRAM). If many pending texture requests all want the same or nearby texels (as they often do), then you get a lot of efficiency here, as you can satisfy many pending requests with only a few memory transactions.  All these operations are pipelined, so while the texture unit is waiting for memory on one batch it can be doing the addressing math for another batch of requests, and so on.</p>

<p>Once the data comes back, the texture unit will decode compressed formats, do sRGB conversion and filtering as necessary, then return the results back to the shader core.</p>

CC BY-SA 4.0
2020-08-04T21:49:24.110Z
@NathanReed Good answer. But "converting UVs to texel coordinates" What? A UV IS a texture coordinate, which is used to get the texel texture[u][v].
KeyC0de
CC BY-SA 4.0
2020-08-04T22:52:02.020Z
@Nikos Not quite. UVs are in the [0,1] range and have to be converted to find the set of actual texels needed, given the texture size, wrap mode, and filtering mode.
Nathan Reed
CC BY-SA 3.0
2015-08-28T18:32:24.263Z
Thanks! That was really helpful. Regading the constants/uniform cache. Are they any tips besides precision (mediump, lowp) I could use to improve uniforms cache hit ratio? Does the order in which I declare uniforms make any difference (as for packing more tightly)?
Felipe Lira
CC BY-SA 3.0
2015-08-28T20:23:42.223Z
@PhilLira Packing can make a difference, yeah. The compiler will insert padding to prevent vectors from being split across 16-byte boundaries, so try to avoid that. I don't think mediump/lowp actually does anything on uniforms, at least on desktop GPUs (maybe it does on mobile). I wouldn't worry too much about uniform cache hit ratio though. That's extremely rarely, if ever, a bottleneck.
Nathan Reed
CC BY-SA 3.0
2015-08-28T06:49:13.630Z
2015-08-28T06:49:13.630Z
Nathan Reed
<p>Modern GPUs have many parallel shading units. I'd like to know how texture cache is managed in that scenario. </p>

<texture><gpu><shader>
How does Texture Cache work considering multiple shader units
Felipe Lira
# How does Texture Cache work considering multiple shader units

(Asked by: username_0 on 2015-08-26)

Modern GPUs have many parallel shading units. I'd like to know how texture cache is managed in that scenario.

**Tags:** texture, gpu, shader


---

## 1 answers

### Answer by username_1 on 2015-08-28. Score: 17 | Accepted answer

At the top level, a GPU is subdivided into a number of shader cores. A small GPU in a notebook or tablet may have only a few cores while a high-end desktop GPU may have dozens.

In addition to the shader cores there are also texture units. They may be grouped together with one texture unit per shader core, or one texture unit shared among two or three shader cores, depending on the GPU.

The whole chip shares a single L2 cache, but the different units will have individual L1 caches. Texture units have texture caches, and shader units have caches for instructions and constants/uniforms, and maybe a separate cache for buffer data depending on whether buffer loads are a separate path from texture loads or not (varies by GPU architecture).

Texture units operate independently and asynchronously from shader cores. When a shader performs a texture read, it sends a request to the texture unit across a little bus between them; the shader can then continue executing if possible, or it may get suspended and allow other shader threads to run while it waits for the texture read to finish.

The texture unit batches up a bunch of requests and performs the addressing math on them—selecting mip levels and anisotropy, converting UVs to texel coordinates, applying clamp/wrap modes, etc. Once it knows which texels it needs, it reads them through the cache hierarchy, the same way that memory reads work on a CPU (look in L1 first, if not there then L2, then DRAM). If many pending texture requests all want the same or nearby texels (as they often do), then you get a lot of efficiency here, as you can satisfy many pending requests with only a few memory transactions. All these operations are pipelined, so while the texture unit is waiting for memory on one batch it can be doing the addressing math for another batch of requests, and so on.

Once the data comes back, the texture unit will decode compressed formats, do sRGB conversion and filtering as necessary, then return the results back to the shader core.

> **Comments:**
> **username_0**: Thanks! That was really helpful. Regading the constants/uniform cache. Are they any tips besides precision (mediump, lowp) I could use to improve uniforms cache hit ratio? Does the order in which I declare uniforms make any difference (as for packing more tightly)?  
> **username_1**: @PhilLira Packing can make a difference, yeah. The compiler will insert padding to prevent vectors from being split across 16-byte boundaries, so try to avoid that. I don't think mediump/lowp actually does anything on uniforms, at least on desktop GPUs (maybe it does on mobile). I wouldn't worry too much about uniform cache hit ratio though. That's extremely rarely, if ever, a bottleneck.  
> **username_2**: @username_1 Good answer. But "converting UVs to texel coordinates" What? A UV IS a texture coordinate, which is used to get the texel texture[u][v].  
> **username_1**: @Nikos Not quite. UVs are in the [0,1] range and have to be converted to find the set of actual texels needed, given the texture size, wrap mode, and filtering mode.  

---

