<p>Virtual Texturing is the logical extreme of texture atlases.</p>

<p><br/></p>

<p>A texture atlas is a single giant texture that contains textures for individual meshes inside it:</p>

<p><img src="https://i.sstatic.net/2h1hm.jpg" alt="Texture Atlas Example"></p>

<p>Texture atlases became popular due to the fact that changing textures causes a full pipeline flush on the GPU. When creating the meshes, the UVs are compressed/shifted so that they represent the correct 'portion' of the whole texture atlas. </p>

<p>As @nathan-reed mentioned in the comments, one of the main drawbacks of texture atlases is losing wrap modes such as repeat, clamp, border, etc. In addition, if the textures don't have enough border around them, you can accidentally sample from an adjacent texture when doing filtering. This can lead to bleeding artifacts.</p>

<p>Texture Atlases do have one major limitation: size. Graphics APIs place a soft limit on how big a texture can be. That said, graphics memory is only so big. So there is also a hard limit on texture size, given by the size of your v-ram. Virtual textures solve this problem, by borrowing concepts from <a href="https://en.wikipedia.org/wiki/Virtual_memory" rel="noreferrer">virtual memory</a>. </p>

<p>Virtual textures exploit the fact that in most scenes, you only see a small portion of all the textures. So, only that subset of textures need to be in vram. The rest can be in main RAM, or on disk.</p>

<p>There are a few ways to implement it, but I will explain the implementation described by Sean Barrett in his <a href="https://www.youtube.com/watch?v=MejJL87yNgI" rel="noreferrer">GDC talk</a>. (which I highly recommend watching)</p>

<p>We have three main elements: the virtual texture, the physical texture, and the lookup table.</p>

<p><img src="https://i.sstatic.net/SFNvJ.jpg" alt="Virtual Texture"></p>

<p>The virtual texture represents the theoretical mega atlas we would have if we had enough vram to fit everything. It doesn't actually exist in memory anywhere. The physical texture represents what pixel data we actually have in vram. The lookup table is the mapping between the two. For convenience, we break all three elements into equal sized tiles, or pages. </p>

<p>The lookup table stores the location of the top-left corner of the tile in the physical texture. So, given a UV to the entire virtual texture, how do we get the corresponding UV for the physical texture?</p>

<p>First, we need to find the location of the page within the physical texture. Then we need to calculate the location of the UV within the page. Finally we can add these two offsets together to get the location of the UV within the physical texture</p>

<pre><code>float2 pageLocInPhysicalTex = ...
float2 inPageLocation = ...
float2 physicalTexUV = pageLocationInPhysicalTex + inPageLocation;
</code></pre>

<p><br/></p>

<p><strong>Calculating pageLocInPhysicalTex</strong> </p>

<p>If we make the lookup table the same size as the number of tiles in the virtual texture, we can just sample the lookup table with nearest neighbor sampling and we will get the location of the top-left corner of the page within the physical texture.</p>

<pre><code>float2 pageLocInPhysicalTex = lookupTable.Sample(virtTexUV, nearestNeighborSampler);
</code></pre>

<p><br /></p>

<p><strong>Calculating inPageLocation</strong></p>

<p>inPageLocation is a UV coordinate that is relative to the top-left of the page, rather than to the top-left of the whole texture. </p>

<p>One way to calculate this is by subtracting off the UV of the top left of the page, then scaling to the size of the page. However, this is quite a bit of math. Instead, we can exploit how IEEE floating point is represented. IEEE floating point stores the fractional part of a number by a series of base 2 fractions.</p>

<p><a href="https://i.sstatic.net/48VfH.png" rel="noreferrer"><img src="https://i.sstatic.net/48VfH.png" alt="enter image description here"></a></p>

<p>In this example, the number is:</p>

<pre><code>number = 0 + (1/2) + (1/8) + (1/16) = 0.6875
</code></pre>

<p>Now lets look at a simplified version of the virtual texture:</p>

<p><img src="https://i.sstatic.net/Jj8K9.png" alt="Simple Virtual Texture"></p>

<p>The 1/2 bit tells us if we're in the left half of the texture or the right. The 1/4 bit tells us which quarter of the half we're in. In this example, since the texture is split into 16, or 4 to a side, these first two bits tell us what page we're in. The remaining bits tell us the location inside the page.</p>

<p>We can get the remaining bits by shifting the float with exp2() and stripping them out with fract()</p>

<pre><code>float2 inPageLocation = virtTexUV * exp2(sqrt(numTiles));
inPageLocation = fract(inPageLocation);
</code></pre>

<p>Where numTiles is a int2 giving the number of tiles per side of the texture. In our example, this would be (4, 4)</p>

<p>So let's calculate the inPageLocation for the green point, (x,y) = (0.6875, 0.375)</p>

<pre><code>inPageLocation = float2(0.6875, 0.375) * exp2(sqrt(int2(4, 4));
               = float2(0.6875, 0.375) * int2(2, 2);
               = float2(1.375, 0.75);

inPageLocation = fract(float2(1.375, 0.75));
               = float2(0.375, 0.75);
</code></pre>

<p>One last thing to do before we're done. Currently, inPageLocation is a UV coordinate in the virtual texture 'space'. However, we want a UV coordinate in the physical texture 'space'. To do this we just have to scale inPageLocation by the ratio of virtual texture size to physical texture size</p>

<pre><code>inPageLocation *= physicalTextureSize / virtualTextureSize;
</code></pre>

<p><br/>
<br/></p>

<p>So the finished function is:</p>

<pre><code>float2 CalculatePhysicalTexUV(float2 virtTexUV, Texture2D&lt;float2&gt; lookupTable, uint2 physicalTexSize, uint2 virtualTexSize, uint2 numTiles) {
    float2 pageLocInPhysicalTex = lookupTable.Sample(virtTexUV, nearestNeighborSampler);

    float2 inPageLocation = virtTexUV * exp2(sqrt(numTiles));
    inPageLocation = fract(inPageLocation);
    inPageLocation *= physicalTexSize / virtualTexSize;

    return pageLocInPhysicalTex + inPageLocation;
}
</code></pre>

CC BY-SA 3.0
2015-12-02T18:18:08.880Z
I'm not, I'm referring to virtual texturing, most well known as idTech 5's [MegaTexture](https://en.wikipedia.org/wiki/MegaTexture) technology. Also see [this](http://holger.dammertz.org/stuff/notes_VirtualTexturing.html) and [this](http://silverspaceship.com/src/svt/). I've seen it mentioned in overview of many modern engines' rendering pipelines, and in a few papers that use a similar approach for shadowmaps. It does have a lot in common with texture atlases, yes, it uses them, in a way, but I'm not confusing it with texture atlases.
Llamageddon
CC BY-SA 3.0
2015-12-02T18:49:41.500Z
Ahh. Thanks for the links. Can you add them to the question. I will update my answer accordingly
RichieSams
CC BY-SA 3.0
2015-12-02T20:58:17.627Z
IMO, the main drawback of simple texture atlases (not virtual textures) is you lose wrap modes like repeat and clamp, and bleeding occurs due to filtering/mipmapping - not floating-point precision. I'd be surprised to see float precision becoming a problem for ordinary (non-virtual) textures; even a 16K texture (the max allowed by current APIs) isn't big enough to really strain float precision.
Nathan Reed
CC BY-SA 3.0
2015-12-03T09:29:11.247Z
@RichieSams Btw, I think your answer is a good one, even if to a different question. You should make a Q&A post.
Llamageddon
CC BY-SA 3.0
2015-12-04T10:29:13.167Z
Hmm, this explains it quite well, though I don't really understand how it works with mip levels. I wish I could write down my specific problem with understanding it down, but it kinda eludes me...
Llamageddon
CC BY-SA 3.0
2018-01-12T03:13:27.813Z
@NathanReed Why would you need the "repeat" mode if you have a quasy unlimited amount of texture space?
Tara
CC BY-SA 4.0
2019-01-02T07:23:42.313Z
@Tara if you have a floor of tiles for example, you would make them repeat rather than have a specific texture for every single tile on the floor. Now you can use the remaining texture space for other things or save texture being send to the graphics card.
@RichieSams `Texture atlases became popular due to the fact that changing textures causes a full pipeline flush on the GPU.` Does this still hold true?
Tare
CC BY-SA 4.0
2019-01-07T07:20:47.280Z
@Tare Yes, but what's the point of using virtual texturing then? The idea is to not make every tile look the same, without resorting to "hacks" like placing decals everywhere. Also instead of using "repeat" addressing, you can just adjust the texture coordinates.
Regarding texture atlases: Yes, changing textures is very expensive on the CPU. That's in fact currently one of the most time consuming things my renderer does. Particles for example are traditionally done with texture atlases exactly because you don't want to swap textures all the time, which can be very expensive.
Tara
CC BY-SA 4.0
2019-01-07T07:25:48.957Z
@Tara i don't think virtual textures are used (only) for that purpose, but for all textures you can currently see in your scene. have a look at this: http://www.adriancourreges.com/blog/2016/09/09/doom-2016-graphics-study/ there is a small part about the virtual texture: `Of course depending on where the player is looking at, this set is going to change: new models will appear on screen`
Tare
CC BY-SA 4.0
2019-01-11T03:06:16.950Z
@Tare I don't know what you are getting at. Yes, if you look around, new tiles become visible and have to be streamed in. Yes, tiling a texture (by repeating texture coordinates or maybe by using texture views) is going to use less tiles. The point of virtual texturing is to be able to have massively more high-res textures by having only what's visible resident in memory. It also makes rendering more efficient, as less texture binding operations are required for rendering (at the cost of texture streaming and decoding). Id used mega textures in Rage in order to give their artists more freedom.
Tara
CC BY-SA 4.0
2019-01-11T07:20:43.947Z
What I was aiming at, was that you seem to think of virtual texturing as being used only for static environment, whereas I understand it to be used for all objects in the world. And then (to get back to your original "repeat" question) you could use your resources for streaming new textures in more efficiently for these other objects like enemies rather than streaming the same tile over and over again.
Tare
CC BY-SA 4.0
2015-12-02T18:08:11.140Z
2018-12-30T21:49:30.093Z
2018-12-30T21:49:30.093Z
RichieSams
<h2>Overview</h2>

<p>The main reason for Virtual Texturing (VT), or <em>Sparse Virtual Textures</em>, as it is sometimes called, is as a memory optimization. The gist of the thing is to only move into video memory the actual texels (generalized as pages/tiles) that you might need for a rendered frame. So it will allow you to have much more texture data in offline or slow storage (HDD, Optical-Disk, Cloud) than it would otherwise fit on video memory or even main memory. If you understand the concept of <a href="https://en.wikipedia.org/wiki/Virtual_memory" rel="noreferrer">Virtual Memory</a> used by modern Operating Systems, it is the same thing in its essence (the name is not given by accident).</p>

<p>VT does not require recomputing UVs in the sense that you'd do that each frame before rendering a mesh, then resubmit vertex data, but it does require some substantial work in the Vertex and Fragment shaders to perform the indirect lookup from the incoming UVs. In a good implementation however, it should be completely transparent for the application if it is using a virtual texture or a traditional one. Actually, most of the time an application will mix both types of texturing, virtual and traditional.</p>

<p>Batching can in theory work very well, though I have never looked into the details of this. Since the usual criteria for grouping geometry are the textures, and with VT, every polygon in the scene can share the same "infinitely large" texture, theoretically, you could achieve full scene drawing with 1 draw call. But in reality, other factors come into play making this impractical.</p>

<h2>Issues with VT</h2>

<p>Zooming in/out and abrupt camera movement are the hardest things to handle in a VT setup. It can look very appealing for a static scene, but once things start moving, more texture pages/tiles will be requested than you can stream for external storage. Async file IO and threading can help, but if it is a real-time system, like in a game, you'll just have to render for a few frames with lower resolution tiles until the hi-res ones arrive, every now and then, resulting in a blurry texture. There's no silver bullet here and that's the biggest issue with the technique, IMO.</p>

<p>Virtual Texturing also doesn't handle transparency in an easy way, so transparent polygons need a separate traditional rendering path for them.</p>

<p>All in all, VT is interesting, but I wouldn't recommend it for everyone. It can work well, but it is hard to implement and optimize, plus there's just too many corner cases and case-specific tweaks needed for my taste. But for large open-world games or data visualization apps, it might be the only possible approach to fit all the content into the available hardware. With a lot of work, it can be made to run fairly efficiently even on limited hardware, like we can see in the PS3 and XBOX360 versions of id's <em>Rage</em>. </p>

<h2>Implementation</h2>

<p>I have managed to get VT working on iOS with OpenGL-ES, to a certain degree. My implementation is not "shippable", but I could conceivably make it so if I wanted and had the resources. <a href="https://bitbucket.org/glampert/vt-mobile" rel="noreferrer">You can view the source code here</a>, it might help getting a better idea of how the pieces fit together. <a href="https://youtu.be/sWz45m0QKj4" rel="noreferrer">Here's a video</a> of a demo running on the iOS Sim. It looks very laggy because the simulator is terrible at emulating shaders, but it runs smoothly on a device.</p>

<p>The following diagram outlines the main components of the system in my implementation. It differs quite a bit from Sean's SVT demo (link down bellow), but it is closer in architecture to the one presented by the paper <em>Accelerating Virtual Texturing Using CUDA</em>, found in the first GPU Pro book (link bellow).</p>

<p><a href="https://i.sstatic.net/F3liK.jpg" rel="noreferrer"><img src="https://i.sstatic.net/F3liK.jpg" alt="virtual texturing system"></a></p>

<ul>
<li><p><code>Page Files</code> are the virtual textures, already cut into tiles (AKA pages) as a preprocessing step, so they are ready to be moved from disk into video memory whenever needed. A page file also contains the whole set of mipmaps, also called <em>virtual mipmaps</em>.</p></li>
<li><p><code>Page Cache Manager</code> keeps an application-side representation of the <code>Page Table</code> and <code>Page Indirection</code> textures. Since moving a page from offline storage to memory is expensive, we need a cache to avoid reloading what is already available. This cache is a very simple <em>Least Recently Used</em> (LRU) cache. The cache is also the component responsible for keeping the physical textures up-to-date with its own local representation of the data.</p></li>
<li><p>The <code>Page Provider</code> is an async job queue that will fetch the pages needed for a given view of the scene and send them to the cache.</p></li>
<li><p>The <code>Page Indirection</code> texture is a texture with one pixel for each page/tile in the virtual texture, that will map the incoming UVs to the <code>Page Table</code> cache texture that has the actual texel data. This texture can get quite large, so it must use some compact format, like RGBA 8:8:8:8 or RGB 5:6:5.</p></li>
</ul>

<p>But we are still missing a key piece here, and that's how to determine which pages must be loaded from storage into the cache and consequently into the <code>Page Table</code>. That's where the <em>Feedback Pass</em> and the <code>Page Resolver</code> enter.</p>

<p>The Feedback Pass is a pre-render of the view, with a custom shader and at a much lower resolution, that will write the ids of the required pages to the color framebuffer. That colorful patchwork of the cube and sphere above are actual page indexes encoded as an RGBA color. This pre-pass rendering is then read into main memory and processed by the <code>Page Resolver</code> to decode the page indexes and fire the new requests with the <code>Page Provider</code>.</p>

<p>After the Feedback pre-pass, the scene can be rendered normally with the VT lookup shaders. But note that we don't wait for new page request to finish, that would be terrible, because we'd simply block on synchronous file IO. The requests are asynchronous and might or might not be ready by the time the final view is rendered. If they are ready, sweet, but if not, we always keep a locked page of a low-res mipmap in the cache as a fallback, so we have some texture data in there to use, but it is going to be blurry.</p>

<h2>Others resources worth checking out</h2>

<ul>
<li><p>The first book in the <a href="http://gpupro.blogspot.com.br/" rel="noreferrer">GPU Pro series</a>. It has two very good articles on VT.</p></li>
<li><p>Mandatory paper by MrElusive: <a href="http://www.mrelusive.com/publications/papers/Software-Virtual-Textures.pdf" rel="noreferrer">Software Virtual Textures</a>.</p></li>
<li><p>The Crytek paper by Martin Mittring: <a href="http://developer.amd.com/wordpress/media/2013/01/Chapter02-Mittring-Advanced_Virtual_Texture_Topics.pdf" rel="noreferrer">Advanced Virtual Texture Topics</a>.</p></li>
<li><p>And <a href="http://silverspaceship.com/src/svt/" rel="noreferrer">Sean's presentation on youtube</a>, which I see you've already found. </p></li>
</ul>

<p>VT is still a somewhat hot topic on Computer Graphics, so there's tons of good material available, you should be able to find a lot more. If there's anything else I can add to this answer, please feel free to ask. I'm a bit rusty on the topic, haven't read much about it for the past year, but it is alway good for the memory to revisit stuff :)</p>

CC BY-SA 3.0
2015-12-03T09:33:14.113Z
Hey, thank you for the excellent answer. I know this is typically frowned upon, but I have various issues, so I mostly just skim through things - to get an intuitive overview of topics for the future(I'm afraid properly learning and implementing things is out of my reach for the moment) - anyway, if possible, could you post a pseudocode example outlining the process itself, ideally, but not necessarily, illustrated?
Llamageddon
CC BY-SA 3.0
2015-12-03T18:47:15.340Z
@Llamageddon, it just so happens that I still had a diagram at hand ;) I'm afraid pseudo-code is going to bit a bit hard to provide, since there's quite a bit of real code to it. But I hope the expanded answer helps giving a general idea of the technique.
glampert
CC BY-SA 3.0
2015-12-03T22:03:09.053Z
Amazing answer, though I still find some details unclear: If the pre-pass is low-res, isn't it possible to miss some textures altogether? What happens then? How do the shaders for the pre-pass and final render look? Is the pre-pass used only for fetching the textures, or...?
Llamageddon
CC BY-SA 3.0
2015-12-04T02:32:18.077Z
It's worth noting that most modern hardware now exposes programmable page tables, eliminating the need for a redirection texture.  This is exposed through e.g. [tag:directx12] [reserved resources](https://msdn.microsoft.com/en-us/library/windows/desktop/dn899181(v=vs.85).aspx), which builds on [tag:directx11] [tiled resources](https://msdn.microsoft.com/en-us/library/windows/desktop/dn786477(v=vs.85).aspx), or [tag:opengl] [sparse textures](https://www.opengl.org/registry/specs/ARB/sparse_texture.txt).
MooseBoys
CC BY-SA 3.0
2015-12-04T03:15:23.287Z
@Llamageddon, the feedback pre-pass can be done at a lower res to save as much computing and memory as possible, since pixels for a page will generally repeat (you can notice the big colored squares in my demo). You're correct that it might eventually miss a visible page like that, but that's not usually going to have a big visual impact because the system should always keep at least the lowest mipmap of the whole VT available in cache. That second paper I linked has all the shader examples in the appendix, you can also refer to the repo for my own project, they are similar.
glampert
CC BY-SA 3.0
2015-12-04T03:16:55.707Z
The feedback pre-pass is only useful for determining the visible set of pages needed for a view/frame, but you could also combine something like depth pre-pass to it.
glampert
CC BY-SA 3.0
2015-12-04T19:52:20.630Z
BTW, why do you say that "Virtual Texturing also doesn't handle transparency in an easy way"? Transparency has nothing to do with how textures are stored in memory...
Nathan Reed
CC BY-SA 3.0
2015-12-05T00:00:57.850Z
@NathanReed, it's because of the page id pre-pass. It's impossible to handle transparency there, because blending 2 or more colors would produce a different page number. What I mean is, suppose object A gets assigned page X which when encoded into a color makes a red tone, now object B gets assigned page Y, which results on green when encoded in the feedback pass. If A and B were to get blended, the color output in the pre-pass for those pixels would be a shade of yellow, translating to the wrong page id/number.
glampert
CC BY-SA 3.0
2015-12-05T02:12:20.280Z
@glampert Ahh, I see; that makes sense. Still, I think there are lots of options for handling transparencies; in the page ID pass, you could dither (so histogramming would see all the pages, unless there were a huge number of transparent layers), or use a [k-buffer approach](http://www.sci.utah.edu/~csilva/papers/i3d2007.pdf), or even just base transparent texture residency on which objects are near the camera (as opposed to rendering them in a feedback pass).
Nathan Reed
CC BY-SA 3.0
2015-12-03T04:05:20.233Z
2015-12-03T18:43:24.437Z
2015-12-03T18:43:24.437Z
glampert
<p>For reference, what I'm referring to is the "generic name" for the technique first(I believe) introduced with idTech 5's <a href="https://en.wikipedia.org/wiki/MegaTexture">MegaTexture</a> technology. See the video <a href="http://holger.dammertz.org/stuff/notes_VirtualTexturing.html">here</a> for a quick glance on how it works.</p>

<p>I've been skimming some papers and publications related to it lately, and what I don't understand is how it can possibly be efficient. Doesn't it require constant recalculation of UV coordinates from the "global texture page" space into the virtual texture coordinates? And how doesn't that curb most attempts at batching geometry altogether? How can it allow arbitrary zooming in? Wouldn't it at some point require subdividing polygons?</p>

<p>There just is so much I don't understand, and I have been unable to find any actually easily approachable resources on the topic.</p>

<rendering><texture><optimisation>
How can virtual texturing actually be efficient?
Llamageddon
# How can virtual texturing actually be efficient?

(Asked by: username_0 on 2015-12-02)

For reference, what I'm referring to is the "generic name" for the technique first(I believe) introduced with idTech 5's [MegaTexture](https://en.wikipedia.org/wiki/MegaTexture) technology. See the video [here](http://holger.dammertz.org/stuff/notes_VirtualTexturing.html) for a quick glance on how it works.

I've been skimming some papers and publications related to it lately, and what I don't understand is how it can possibly be efficient. Doesn't it require constant recalculation of UV coordinates from the "global texture page" space into the virtual texture coordinates? And how doesn't that curb most attempts at batching geometry altogether? How can it allow arbitrary zooming in? Wouldn't it at some point require subdividing polygons?

There just is so much I don't understand, and I have been unable to find any actually easily approachable resources on the topic.

**Tags:** rendering, texture, optimisation


---

## 2 answers

### Answer by username_5 on 2015-12-03. Score: 34 | Accepted answer

## Overview

The main reason for Virtual Texturing (VT), or *Sparse Virtual Textures*, as it is sometimes called, is as a memory optimization. The gist of the thing is to only move into video memory the actual texels (generalized as pages/tiles) that you might need for a rendered frame. So it will allow you to have much more texture data in offline or slow storage (HDD, Optical-Disk, Cloud) than it would otherwise fit on video memory or even main memory. If you understand the concept of [Virtual Memory](https://en.wikipedia.org/wiki/Virtual_memory) used by modern Operating Systems, it is the same thing in its essence (the name is not given by accident).

VT does not require recomputing UVs in the sense that you'd do that each frame before rendering a mesh, then resubmit vertex data, but it does require some substantial work in the Vertex and Fragment shaders to perform the indirect lookup from the incoming UVs. In a good implementation however, it should be completely transparent for the application if it is using a virtual texture or a traditional one. Actually, most of the time an application will mix both types of texturing, virtual and traditional.

Batching can in theory work very well, though I have never looked into the details of this. Since the usual criteria for grouping geometry are the textures, and with VT, every polygon in the scene can share the same "infinitely large" texture, theoretically, you could achieve full scene drawing with 1 draw call. But in reality, other factors come into play making this impractical.

## Issues with VT

Zooming in/out and abrupt camera movement are the hardest things to handle in a VT setup. It can look very appealing for a static scene, but once things start moving, more texture pages/tiles will be requested than you can stream for external storage. Async file IO and threading can help, but if it is a real-time system, like in a game, you'll just have to render for a few frames with lower resolution tiles until the hi-res ones arrive, every now and then, resulting in a blurry texture. There's no silver bullet here and that's the biggest issue with the technique, IMO.

Virtual Texturing also doesn't handle transparency in an easy way, so transparent polygons need a separate traditional rendering path for them.

All in all, VT is interesting, but I wouldn't recommend it for everyone. It can work well, but it is hard to implement and optimize, plus there's just too many corner cases and case-specific tweaks needed for my taste. But for large open-world games or data visualization apps, it might be the only possible approach to fit all the content into the available hardware. With a lot of work, it can be made to run fairly efficiently even on limited hardware, like we can see in the PS3 and XBOX360 versions of id's *Rage*.

## Implementation

I have managed to get VT working on iOS with OpenGL-ES, to a certain degree. My implementation is not "shippable", but I could conceivably make it so if I wanted and had the resources. [You can view the source code here](https://bitbucket.org/username_5/vt-mobile), it might help getting a better idea of how the pieces fit together. [Here's a video](https://youtu.be/sWz45m0QKj4) of a demo running on the iOS Sim. It looks very laggy because the simulator is terrible at emulating shaders, but it runs smoothly on a device.

The following diagram outlines the main components of the system in my implementation. It differs quite a bit from Sean's SVT demo (link down bellow), but it is closer in architecture to the one presented by the paper *Accelerating Virtual Texturing Using CUDA*, found in the first GPU Pro book (link bellow).

[![virtual texturing system](https://i.sstatic.net/F3liK.jpg)](https://i.sstatic.net/F3liK.jpg)

* `Page Files` are the virtual textures, already cut into tiles (AKA pages) as a preprocessing step, so they are ready to be moved from disk into video memory whenever needed. A page file also contains the whole set of mipmaps, also called *virtual mipmaps*.
* `Page Cache Manager` keeps an application-side representation of the `Page Table` and `Page Indirection` textures. Since moving a page from offline storage to memory is expensive, we need a cache to avoid reloading what is already available. This cache is a very simple *Least Recently Used* (LRU) cache. The cache is also the component responsible for keeping the physical textures up-to-date with its own local representation of the data.
* The `Page Provider` is an async job queue that will fetch the pages needed for a given view of the scene and send them to the cache.
* The `Page Indirection` texture is a texture with one pixel for each page/tile in the virtual texture, that will map the incoming UVs to the `Page Table` cache texture that has the actual texel data. This texture can get quite large, so it must use some compact format, like RGBA 8:8:8:8 or RGB 5:6:5.

But we are still missing a key piece here, and that's how to determine which pages must be loaded from storage into the cache and consequently into the `Page Table`. That's where the *Feedback Pass* and the `Page Resolver` enter.

The Feedback Pass is a pre-render of the view, with a custom shader and at a much lower resolution, that will write the ids of the required pages to the color framebuffer. That colorful patchwork of the cube and sphere above are actual page indexes encoded as an RGBA color. This pre-pass rendering is then read into main memory and processed by the `Page Resolver` to decode the page indexes and fire the new requests with the `Page Provider`.

After the Feedback pre-pass, the scene can be rendered normally with the VT lookup shaders. But note that we don't wait for new page request to finish, that would be terrible, because we'd simply block on synchronous file IO. The requests are asynchronous and might or might not be ready by the time the final view is rendered. If they are ready, sweet, but if not, we always keep a locked page of a low-res mipmap in the cache as a fallback, so we have some texture data in there to use, but it is going to be blurry.

## Others resources worth checking out

* The first book in the [GPU Pro series](http://gpupro.blogspot.com.br/). It has two very good articles on VT.
* Mandatory paper by MrElusive: [Software Virtual Textures](http://www.mrelusive.com/publications/papers/Software-Virtual-Textures.pdf).
* The Crytek paper by Martin Mittring: [Advanced Virtual Texture Topics](http://developer.amd.com/wordpress/media/2013/01/Chapter02-Mittring-Advanced_Virtual_Texture_Topics.pdf).
* And [Sean's presentation on youtube](http://silverspaceship.com/src/svt/), which I see you've already found.

VT is still a somewhat hot topic on Computer Graphics, so there's tons of good material available, you should be able to find a lot more. If there's anything else I can add to this answer, please feel free to ask. I'm a bit rusty on the topic, haven't read much about it for the past year, but it is alway good for the memory to revisit stuff :)

> **Comments:**
> **username_0**: Hey, thank you for the excellent answer. I know this is typically frowned upon, but I have various issues, so I mostly just skim through things - to get an intuitive overview of topics for the future(I'm afraid properly learning and implementing things is out of my reach for the moment) - anyway, if possible, could you post a pseudocode example outlining the process itself, ideally, but not necessarily, illustrated?  
> **username_5**: @username_0, it just so happens that I still had a diagram at hand ;) I'm afraid pseudo-code is going to bit a bit hard to provide, since there's quite a bit of real code to it. But I hope the expanded answer helps giving a general idea of the technique.  
> **username_0**: Amazing answer, though I still find some details unclear: If the pre-pass is low-res, isn't it possible to miss some textures altogether? What happens then? How do the shaders for the pre-pass and final render look? Is the pre-pass used only for fetching the textures, or...?  
> **username_6**: It's worth noting that most modern hardware now exposes programmable page tables, eliminating the need for a redirection texture.  This is exposed through e.g. [tag:directx12] [reserved resources](https://msdn.microsoft.com/en-us/library/windows/desktop/dn899181(v=vs.85).aspx), which builds on [tag:directx11] [tiled resources](https://msdn.microsoft.com/en-us/library/windows/desktop/dn786477(v=vs.85).aspx), or [tag:opengl] [sparse textures](https://www.opengl.org/registry/specs/ARB/sparse_texture.txt).  
> **username_5**: @username_0, the feedback pre-pass can be done at a lower res to save as much computing and memory as possible, since pixels for a page will generally repeat (you can notice the big colored squares in my demo). You're correct that it might eventually miss a visible page like that, but that's not usually going to have a big visual impact because the system should always keep at least the lowest mipmap of the whole VT available in cache. That second paper I linked has all the shader examples in the appendix, you can also refer to the repo for my own project, they are similar.  
> **username_5**: The feedback pre-pass is only useful for determining the visible set of pages needed for a view/frame, but you could also combine something like depth pre-pass to it.  
> **username_2**: BTW, why do you say that "Virtual Texturing also doesn't handle transparency in an easy way"? Transparency has nothing to do with how textures are stored in memory...  
> **username_5**: @username_2, it's because of the page id pre-pass. It's impossible to handle transparency there, because blending 2 or more colors would produce a different page number. What I mean is, suppose object A gets assigned page X which when encoded into a color makes a red tone, now object B gets assigned page Y, which results on green when encoded in the feedback pass. If A and B were to get blended, the color output in the pre-pass for those pixels would be a shade of yellow, translating to the wrong page id/number.  
> **username_2**: @username_5 Ahh, I see; that makes sense. Still, I think there are lots of options for handling transparencies; in the page ID pass, you could dither (so histogramming would see all the pages, unless there were a huge number of transparent layers), or use a [k-buffer approach](http://www.sci.utah.edu/~csilva/papers/i3d2007.pdf), or even just base transparent texture residency on which objects are near the camera (as opposed to rendering them in a feedback pass).  

---

### Answer by username_1 on 2015-12-02. Score: 15

Virtual Texturing is the logical extreme of texture atlases.

A texture atlas is a single giant texture that contains textures for individual meshes inside it:

![Texture Atlas Example](https://i.sstatic.net/2h1hm.jpg)

Texture atlases became popular due to the fact that changing textures causes a full pipeline flush on the GPU. When creating the meshes, the UVs are compressed/shifted so that they represent the correct 'portion' of the whole texture atlas.

As @username_2-username_2 mentioned in the comments, one of the main drawbacks of texture atlases is losing wrap modes such as repeat, clamp, border, etc. In addition, if the textures don't have enough border around them, you can accidentally sample from an adjacent texture when doing filtering. This can lead to bleeding artifacts.

Texture Atlases do have one major limitation: size. Graphics APIs place a soft limit on how big a texture can be. That said, graphics memory is only so big. So there is also a hard limit on texture size, given by the size of your v-ram. Virtual textures solve this problem, by borrowing concepts from [virtual memory](https://en.wikipedia.org/wiki/Virtual_memory).

Virtual textures exploit the fact that in most scenes, you only see a small portion of all the textures. So, only that subset of textures need to be in vram. The rest can be in main RAM, or on disk.

There are a few ways to implement it, but I will explain the implementation described by Sean Barrett in his [GDC talk](https://www.youtube.com/watch?v=MejJL87yNgI). (which I highly recommend watching)

We have three main elements: the virtual texture, the physical texture, and the lookup table.

![Virtual Texture](https://i.sstatic.net/SFNvJ.jpg)

The virtual texture represents the theoretical mega atlas we would have if we had enough vram to fit everything. It doesn't actually exist in memory anywhere. The physical texture represents what pixel data we actually have in vram. The lookup table is the mapping between the two. For convenience, we break all three elements into equal sized tiles, or pages.

The lookup table stores the location of the top-left corner of the tile in the physical texture. So, given a UV to the entire virtual texture, how do we get the corresponding UV for the physical texture?

First, we need to find the location of the page within the physical texture. Then we need to calculate the location of the UV within the page. Finally we can add these two offsets together to get the location of the UV within the physical texture

```
float2 pageLocInPhysicalTex = ...
float2 inPageLocation = ...
float2 physicalTexUV = pageLocationInPhysicalTex + inPageLocation;

```

**Calculating pageLocInPhysicalTex**

If we make the lookup table the same size as the number of tiles in the virtual texture, we can just sample the lookup table with nearest neighbor sampling and we will get the location of the top-left corner of the page within the physical texture.

```
float2 pageLocInPhysicalTex = lookupTable.Sample(virtTexUV, nearestNeighborSampler);

```

**Calculating inPageLocation**

inPageLocation is a UV coordinate that is relative to the top-left of the page, rather than to the top-left of the whole texture.

One way to calculate this is by subtracting off the UV of the top left of the page, then scaling to the size of the page. However, this is quite a bit of math. Instead, we can exploit how IEEE floating point is represented. IEEE floating point stores the fractional part of a number by a series of base 2 fractions.

[![enter image description here](https://i.sstatic.net/48VfH.png)](https://i.sstatic.net/48VfH.png)

In this example, the number is:

```
number = 0 + (1/2) + (1/8) + (1/16) = 0.6875

```

Now lets look at a simplified version of the virtual texture:

![Simple Virtual Texture](https://i.sstatic.net/Jj8K9.png)

The 1/2 bit tells us if we're in the left half of the texture or the right. The 1/4 bit tells us which quarter of the half we're in. In this example, since the texture is split into 16, or 4 to a side, these first two bits tell us what page we're in. The remaining bits tell us the location inside the page.

We can get the remaining bits by shifting the float with exp2() and stripping them out with fract()

```
float2 inPageLocation = virtTexUV * exp2(sqrt(numTiles));
inPageLocation = fract(inPageLocation);

```

Where numTiles is a int2 giving the number of tiles per side of the texture. In our example, this would be (4, 4)

So let's calculate the inPageLocation for the green point, (x,y) = (0.6875, 0.375)

```
inPageLocation = float2(0.6875, 0.375) * exp2(sqrt(int2(4, 4));
               = float2(0.6875, 0.375) * int2(2, 2);
               = float2(1.375, 0.75);

inPageLocation = fract(float2(1.375, 0.75));
               = float2(0.375, 0.75);

```

One last thing to do before we're done. Currently, inPageLocation is a UV coordinate in the virtual texture 'space'. However, we want a UV coordinate in the physical texture 'space'. To do this we just have to scale inPageLocation by the ratio of virtual texture size to physical texture size

```
inPageLocation *= physicalTextureSize / virtualTextureSize;

```

So the finished function is:

```
float2 CalculatePhysicalTexUV(float2 virtTexUV, Texture2D<float2> lookupTable, uint2 physicalTexSize, uint2 virtualTexSize, uint2 numTiles) {
    float2 pageLocInPhysicalTex = lookupTable.Sample(virtTexUV, nearestNeighborSampler);

    float2 inPageLocation = virtTexUV * exp2(sqrt(numTiles));
    inPageLocation = fract(inPageLocation);
    inPageLocation *= physicalTexSize / virtualTexSize;

    return pageLocInPhysicalTex + inPageLocation;
}

```

> **Comments:**
> **username_0**: I'm not, I'm referring to virtual texturing, most well known as idTech 5's [MegaTexture](https://en.wikipedia.org/wiki/MegaTexture) technology. Also see [this](http://holger.dammertz.org/stuff/notes_VirtualTexturing.html) and [this](http://silverspaceship.com/src/svt/). I've seen it mentioned in overview of many modern engines' rendering pipelines, and in a few papers that use a similar approach for shadowmaps. It does have a lot in common with texture atlases, yes, it uses them, in a way, but I'm not confusing it with texture atlases.  
> **username_1**: Ahh. Thanks for the links. Can you add them to the question. I will update my answer accordingly  
> **username_2**: IMO, the main drawback of simple texture atlases (not virtual textures) is you lose wrap modes like repeat and clamp, and bleeding occurs due to filtering/mipmapping - not floating-point precision. I'd be surprised to see float precision becoming a problem for ordinary (non-virtual) textures; even a 16K texture (the max allowed by current APIs) isn't big enough to really strain float precision.  
> **username_0**: @username_1 Btw, I think your answer is a good one, even if to a different question. You should make a Q&A post.  
> **username_0**: Hmm, this explains it quite well, though I don't really understand how it works with mip levels. I wish I could write down my specific problem with understanding it down, but it kinda eludes me...  
> **username_3**: @username_2 Why would you need the "repeat" mode if you have a quasy unlimited amount of texture space?  
> **username_4**: @username_3 if you have a floor of tiles for example, you would make them repeat rather than have a specific texture for every single tile on the floor. Now you can use the remaining texture space for other things or save texture being send to the graphics card. @username_1 `Texture atlases became popular due to the fact that changing textures causes a full pipeline flush on the GPU.` Does this still hold true?  
> **username_3**: @username_4 Yes, but what's the point of using virtual texturing then? The idea is to not make every tile look the same, without resorting to "hacks" like placing decals everywhere. Also instead of using "repeat" addressing, you can just adjust the texture coordinates. Regarding texture atlases: Yes, changing textures is very expensive on the CPU. That's in fact currently one of the most time consuming things my renderer does. Particles for example are traditionally done with texture atlases exactly because you don't want to swap textures all the time, which can be very expensive.  
> **username_4**: @username_3 i don't think virtual textures are used (only) for that purpose, but for all textures you can currently see in your scene. have a look at this: http://www.adriancourreges.com/blog/2016/09/09/doom-2016-graphics-study/ there is a small part about the virtual texture: `Of course depending on where the player is looking at, this set is going to change: new models will appear on screen`  
> **username_3**: @username_4 I don't know what you are getting at. Yes, if you look around, new tiles become visible and have to be streamed in. Yes, tiling a texture (by repeating texture coordinates or maybe by using texture views) is going to use less tiles. The point of virtual texturing is to be able to have massively more high-res textures by having only what's visible resident in memory. It also makes rendering more efficient, as less texture binding operations are required for rendering (at the cost of texture streaming and decoding). Id used mega textures in Rage in order to give their artists more freedom.  
> **username_4**: What I was aiming at, was that you seem to think of virtual texturing as being used only for static environment, whereas I understand it to be used for all objects in the world. And then (to get back to your original "repeat" question) you could use your resources for streaming new textures in more efficiently for these other objects like enemies rather than streaming the same tile over and over again.  

---

