<p>If the scene does not entirely fit into memory, you are entering the field of out-of-core rendering. There are essentially two approaches here: a) Generate your scene on-demand b) Load your scene on-demand</p>

<p>The former approach aligns well with most animation workflows, where models are heavily subdivided using e.g. Catmull-Clark and can become very memory-intensive, but the base meshes themselves easily fit into memory. Pixar have a few papers about this (e.g. <a href="http://www.pixar.com/companyinfo/research/per/eg03.pdf">Ray Differentials and Multiresolution Geometry Caching
for Distribution Ray Tracing in Complex Scenes</a>), but the gist of it is that models are only subdivided when they are hit by a ray, and only subdivided as much as is reasonable for such a ray (e.g. diffuse interreflection need less accuracy than mirror reflections). The rest is handled by a geometry cache, which keeps the subdivided models in memory and hopefully makes the process efficient by a good eviction strategy.</p>

<p>As long as all your base meshes comfortably fit into memory, you can easily go out-of-core and render meshes at subdivision levels that would never fit into memory. The geometry cache also scales nicely with the amount of memory you have, allowing you to weigh RAM vs. render times. This was also used in <a href="http://graphics.pixar.com/library/RayTracingCars/paper.pdf">Cars</a> I believe.</p>

<p>The second approach is more general and does not rely on heavy use of subdivision. Instead, it relies on the fact that your scene was most likely made by an artist and already comes partitioned into reasonably small objects that fit into memory individually. The idea is then to keep two hierarchies (kD-tree or bounding volume hierarchy): A top-level hierarchy that only stores bounding boxes of the objects in your scene, and a low-level hierarchy that stores the actual geometry. There is one such low-level hierarchy for each object.</p>

<p>In this approach, you ideally already store a bounding box along with each object on disk. As the scene is loaded, you only build the top-level hierarchy initially, meaning you only have to look at the bounding boxes and not the geometry. You then start tracing rays and traverse them through the hierarchy. Whenever a ray hits a leaf node in the top-level hierarchy (i.e. it hits the bounding box of an object), that object is loaded into memory and its low-level hierarchy is built. The ray then continues down into tracing that object. Combined with an object cache that keeps as much of the low-level hierarchy in memory as possible, this can perform reasonably well.</p>

<p>The first benefit of such an approach is that objects that are never hit are never loaded, meaning that it automatically adapts to the visibility in your scene. The second benefit is that if you are tracing many rays, you don't have to load an object immediately as it is hit by a ray; instead, you can hold that ray and wait until enough rays have hit that object, amortizing the load over multiple ray hits.</p>

<p>You can also combine this approach with a ray sorting algorithm such as <a href="https://disney-animation.s3.amazonaws.com/uploads/production/publication_asset/70/asset/Sorted_Deferred_Shading_For_Production_Path_Tracing.pdf">Sorted Deferred Shading for Production Path Tracing</a> to avoid thrashing due to incoherent rays. The mentioned paper describes the architecture of Disney's Hyperion renderer, used for Big Hero 6 I believe, so it most likely can handle scenes at production scale.</p>

CC BY-SA 3.0
2015-08-09T23:41:26.250Z
This is super interesting! So is the Disney paper you linked.
John Calsbeek
CC BY-SA 3.0
2015-08-09T22:47:46.967Z
2015-08-09T22:47:46.967Z
Benedikt Bitterli
<p>If you organize your scene in a spatial structure (the usual way being a <a href="https://en.wikipedia.org/wiki/Bounding_volume_hierarchy">Bounding Volume Hierarchy</a>), you can use a sort of virtual scene (I am making up this term, in reference to <a href="https://developer.nvidia.com/sites/default/files/akamai/gameworks/events/gdc14/GDC_14_Real%20Virtual%20Texturing%20-%20Taking%20Advantage%20of%20DirectX%2011.2%20Tiled%20Resources.pdf">virtual textures</a>).</p>

<p>A memory manager would keep only a limited number of bounding boxes loaded at a time, and abstract the operation consisting in retrieving one.</p>

<p>This way, a box would be loaded only as needed: when a ray hits a bounding box, the box get loaded to resolve the collision. Later on when another box needs to be loaded, the unused one is deleted to make room for the new one.</p>

<p>With all these boxes getting loaded and deleted, the ray coherency would be a major factor in speed. I suppose a further improvement could be to defer loading, by reordering rays to treat first the boxes that are already loaded.</p>

CC BY-SA 3.0
2015-08-09T11:40:33.210Z
Yes something like this.
joojaa
CC BY-SA 3.0
2015-08-09T11:38:54.363Z
2015-08-09T11:38:54.363Z
Julien Guertault
<p>What you do is you load triangles into memory from disk based on what has been hit previously. You can begin with triangles in close proximity first. The reasoning is that in one area the rays are likely to hit same triangles repeatedly. And eventually you will be somewhat efficient. (For this reason it is a good idea to cache last hit triangle in occlusion tracing that dont care of order)</p>

<p>Second you store the triangles in a spatial tree that allows you to do quick searching from the disk, to renew what portions you are having in memory by proximity. So load only branches that will be in the way of the ray. If its some kind of voxel tree, like a octree you can even sort secondary rays and solve them by coherence. A BSP tree is also somewhat good at pruning areas.</p>

<p>There are cases where this fails but its reasonably efficient in most scene buckets if your not rendering noise...</p>

CC BY-SA 3.0
2015-08-09T11:39:04.653Z
2015-08-09T11:59:09.313Z
2015-08-09T11:59:09.313Z
joojaa
<p>The approaches mentioned by the 3 answers up to now (May 2020) mention:</p>

<ul>
<li>subdivision surfaces</li>
<li>BVH per object tracing/eviction + ray cache + memory management</li>
</ul>

<p>There are other orthogonal techniques that can be added or used by themselves. That is decimation based on something else than OpenSubdiv. Like:</p>

<ul>
<li>Distance fields</li>
<li>Octahedral impostors <a href="https://www.shaderbits.com/blog/octahedral-impostors" rel="nofollow noreferrer">https://www.shaderbits.com/blog/octahedral-impostors</a></li>
<li>image meshes (geometry is baked in maps)</li>
<li>camera mapping</li>
<li>voxelization</li>
</ul>

<p>A different angle of thought: if your scene doesn't fit in memory, it has an entropy problem. You only have 4k res (~8 Million pixels?), therefore in the sense of Shannon "more than memory" cannot reasonably map to less data than can be projected to these 8Mp. Which means a scene that uses "more than memory" is most likely a super overkill. Thus again, the simplification approach will probably not only speed up things, but also could help aliasing.</p>

CC BY-SA 4.0
2020-05-25T09:37:10.240Z
I strongly disagree with your interpretation of information theory. It is true, that in the end you will need `#pixels * #channels * #bits_per_channel` but you don't want to guess that data, you want to compute them by evaluating the rendering equation. Note that you have many free variables like camera and light parameters, etc. You don't want to pre-bake an optimized scene for each possible camera position or light setup! For example that would mean to precompute all textures for all possible perspectives.
Isolin
CC BY-SA 4.0
2020-05-25T09:41:06.300Z
Another wrong assumption of yours is perhaps that only the visible objects contribute to the final image. However, there may be shadow casters all around, layers of transparent objects that accumulate together, specular highlights and caustics from behind the camera reflecting towards object in front of the camera and possibly back again. In order to compute all the light bounces correctly, you will easily end up with scenes much more information content (orders of magnitude more) than the resulting image.
Isolin
CC BY-SA 4.0
2020-05-25T09:50:22.390Z
The last bit of additional information you need to take into account is that if you use a spectral renderer, you will try to have more than just RGB data for your materials, but the rendering output will be in most cases just RGB.
Isolin
CC BY-SA 4.0
2020-05-25T10:44:27.143Z
@Isolin Your first remark I would argue, yes we do. That's what we do with mipmaps. When we prepare any sort of map, especially impostors, we do pre-bake them for any sort of camera distance or angle. Any sort of LOD of meshes is exactly that, it's a pre-baked optimization for each possible camera position. Now your second remark is interesting. It made me ponder about the universe. We can say that the faint GI caused by galaxies contribute to the final image. Though I doubt that it's chaotic; meaning they can be reduced to few parameters. But caustics are. good catch
v.oddou
CC BY-SA 4.0
2020-05-25T07:56:21.213Z
2020-05-25T07:56:21.213Z
v.oddou
<p>If the scene to be raytraced cannot be stored in memory, then without adding more RAM to the machine it seems unrealistic to render it in a practical time span, due to the need to load different parts of the scene from disk potentially several times per pixel.</p>

<p>Is there any way around this? I'm trying to think of some way of performing a large number of the calculations involving a particular subset of the scene all at once, to reduce the number of times it needs to be loaded into memory. Is there any other way of improving the speed in such a case?</p>

<raytracing><memory>
How can I raytrace a scene that does not fit into memory?
trichoplax is on Codidact now
# How can I raytrace a scene that does not fit into memory?

(Asked by: username_0 on 2015-08-09)

If the scene to be raytraced cannot be stored in memory, then without adding more RAM to the machine it seems unrealistic to render it in a practical time span, due to the need to load different parts of the scene from disk potentially several times per pixel.

Is there any way around this? I'm trying to think of some way of performing a large number of the calculations involving a particular subset of the scene all at once, to reduce the number of times it needs to be loaded into memory. Is there any other way of improving the speed in such a case?

**Tags:** raytracing, memory


---

## 4 answers

### Answer by username_1 on 2015-08-09. Score: 13 | Accepted answer

If the scene does not entirely fit into memory, you are entering the field of out-of-core rendering. There are essentially two approaches here: a) Generate your scene on-demand b) Load your scene on-demand

The former approach aligns well with most animation workflows, where models are heavily subdivided using e.g. Catmull-Clark and can become very memory-intensive, but the base meshes themselves easily fit into memory. Pixar have a few papers about this (e.g. [Ray Differentials and Multiresolution Geometry Caching
for Distribution Ray Tracing in Complex Scenes](http://www.pixar.com/companyinfo/research/per/eg03.pdf)), but the gist of it is that models are only subdivided when they are hit by a ray, and only subdivided as much as is reasonable for such a ray (e.g. diffuse interreflection need less accuracy than mirror reflections). The rest is handled by a geometry cache, which keeps the subdivided models in memory and hopefully makes the process efficient by a good eviction strategy.

As long as all your base meshes comfortably fit into memory, you can easily go out-of-core and render meshes at subdivision levels that would never fit into memory. The geometry cache also scales nicely with the amount of memory you have, allowing you to weigh RAM vs. render times. This was also used in [Cars](http://graphics.pixar.com/library/RayTracingCars/paper.pdf) I believe.

The second approach is more general and does not rely on heavy use of subdivision. Instead, it relies on the fact that your scene was most likely made by an artist and already comes partitioned into reasonably small objects that fit into memory individually. The idea is then to keep two hierarchies (kD-tree or bounding volume hierarchy): A top-level hierarchy that only stores bounding boxes of the objects in your scene, and a low-level hierarchy that stores the actual geometry. There is one such low-level hierarchy for each object.

In this approach, you ideally already store a bounding box along with each object on disk. As the scene is loaded, you only build the top-level hierarchy initially, meaning you only have to look at the bounding boxes and not the geometry. You then start tracing rays and traverse them through the hierarchy. Whenever a ray hits a leaf node in the top-level hierarchy (i.e. it hits the bounding box of an object), that object is loaded into memory and its low-level hierarchy is built. The ray then continues down into tracing that object. Combined with an object cache that keeps as much of the low-level hierarchy in memory as possible, this can perform reasonably well.

The first benefit of such an approach is that objects that are never hit are never loaded, meaning that it automatically adapts to the visibility in your scene. The second benefit is that if you are tracing many rays, you don't have to load an object immediately as it is hit by a ray; instead, you can hold that ray and wait until enough rays have hit that object, amortizing the load over multiple ray hits.

You can also combine this approach with a ray sorting algorithm such as [Sorted Deferred Shading for Production Path Tracing](https://disney-animation.s3.amazonaws.com/uploads/production/publication_asset/70/asset/Sorted_Deferred_Shading_For_Production_Path_Tracing.pdf) to avoid thrashing due to incoherent rays. The mentioned paper describes the architecture of Disney's Hyperion renderer, used for Big Hero 6 I believe, so it most likely can handle scenes at production scale.

> **Comments:**
> **username_2**: This is super interesting! So is the Disney paper you linked.  

---

### Answer by username_3 on 2015-08-09. Score: 7

If you organize your scene in a spatial structure (the usual way being a [Bounding Volume Hierarchy](https://en.wikipedia.org/wiki/Bounding_volume_hierarchy)), you can use a sort of virtual scene (I am making up this term, in reference to [virtual textures](https://developer.nvidia.com/sites/default/files/akamai/gameworks/events/gdc14/GDC_14_Real%20Virtual%20Texturing%20-%20Taking%20Advantage%20of%20DirectX%2011.2%20Tiled%20Resources.pdf)).

A memory manager would keep only a limited number of bounding boxes loaded at a time, and abstract the operation consisting in retrieving one.

This way, a box would be loaded only as needed: when a ray hits a bounding box, the box get loaded to resolve the collision. Later on when another box needs to be loaded, the unused one is deleted to make room for the new one.

With all these boxes getting loaded and deleted, the ray coherency would be a major factor in speed. I suppose a further improvement could be to defer loading, by reordering rays to treat first the boxes that are already loaded.

> **Comments:**
> **username_4**: Yes something like this.  

---

### Answer by username_4 on 2015-08-09. Score: 1

What you do is you load triangles into memory from disk based on what has been hit previously. You can begin with triangles in close proximity first. The reasoning is that in one area the rays are likely to hit same triangles repeatedly. And eventually you will be somewhat efficient. (For this reason it is a good idea to cache last hit triangle in occlusion tracing that dont care of order)

Second you store the triangles in a spatial tree that allows you to do quick searching from the disk, to renew what portions you are having in memory by proximity. So load only branches that will be in the way of the ray. If its some kind of voxel tree, like a octree you can even sort secondary rays and solve them by coherence. A BSP tree is also somewhat good at pruning areas.

There are cases where this fails but its reasonably efficient in most scene buckets if your not rendering noise...


---

### Answer by username_5 on 2020-05-25. Score: 0

The approaches mentioned by the 3 answers up to now (May 2020) mention:

* subdivision surfaces
* BVH per object tracing/eviction + ray cache + memory management

There are other orthogonal techniques that can be added or used by themselves. That is decimation based on something else than OpenSubdiv. Like:

* Distance fields
* Octahedral impostors <https://www.shaderbits.com/blog/octahedral-impostors>
* image meshes (geometry is baked in maps)
* camera mapping
* voxelization

A different angle of thought: if your scene doesn't fit in memory, it has an entropy problem. You only have 4k res (~8 Million pixels?), therefore in the sense of Shannon "more than memory" cannot reasonably map to less data than can be projected to these 8Mp. Which means a scene that uses "more than memory" is most likely a super overkill. Thus again, the simplification approach will probably not only speed up things, but also could help aliasing.

> **Comments:**
> **username_6**: I strongly disagree with your interpretation of information theory. It is true, that in the end you will need `#pixels * #channels * #bits_per_channel` but you don't want to guess that data, you want to compute them by evaluating the rendering equation. Note that you have many free variables like camera and light parameters, etc. You don't want to pre-bake an optimized scene for each possible camera position or light setup! For example that would mean to precompute all textures for all possible perspectives.  
> **username_6**: Another wrong assumption of yours is perhaps that only the visible objects contribute to the final image. However, there may be shadow casters all around, layers of transparent objects that accumulate together, specular highlights and caustics from behind the camera reflecting towards object in front of the camera and possibly back again. In order to compute all the light bounces correctly, you will easily end up with scenes much more information content (orders of magnitude more) than the resulting image.  
> **username_6**: The last bit of additional information you need to take into account is that if you use a spectral renderer, you will try to have more than just RGB data for your materials, but the rendering output will be in most cases just RGB.  
> **username_5**: @username_6 Your first remark I would argue, yes we do. That's what we do with mipmaps. When we prepare any sort of map, especially impostors, we do pre-bake them for any sort of camera distance or angle. Any sort of LOD of meshes is exactly that, it's a pre-baked optimization for each possible camera position. Now your second remark is interesting. It made me ponder about the universe. We can say that the faint GI caused by galaxies contribute to the final image. Though I doubt that it's chaotic; meaning they can be reduced to few parameters. But caustics are. good catch  

---

