<p>I think you're seeing cache effects. The GPU will begin vertex processing for each strip in sequential order, and in configuration 1 the GPU will fetch a different chunk of memory for each triangle in the strip (since, as you said, the strips themselves are not contiguous in memory in this configuration).</p>

<p>If I understand correctly, each entry in the ring buffer you are referring to is a triangle, so you will have at least 3 vertices adjacent to each other (in configuration 1). This has decent <a href="https://en.wikipedia.org/wiki/Locality_of_reference" rel="nofollow noreferrer">spatial locality</a>, but fetching this memory will probably also pull in memory for adjacent vertices, which won't be immediately useful (since it belongs to a different strip).</p>

<p>In configuration 2, the adjacent vertices <em>will</em> be immediately useful, since they're the next vertices the GPU will need to render for the current triangle strip. The result is, probably, less total memory traffic.</p>

<p>This is a somewhat simplified explanation (for example, it brushes over the parallelism occurring on the GPU), but it seems to match what you're observing. One way to justify the choice of configuration 2 over configuration 1 is that you're making it easier on the GPU at the cost of the CPU—however, the CPU has 1/MAX_AGE of the work to do as the GPU each frame, so optimizing for CPU updating ("a concentrated chunk of memory is updated each frame") will be less valuable.</p>

CC BY-SA 4.0
2018-09-06T00:48:42.270Z
Why would it fetch things in a non sequential order? The index buffer has it go linearly through the structure. So the draw call and index buffer are the same in both scenarios. Sorry if that wasnt clear in the original question. The only way I could see this mattering is if the gpu overrides my index buffer to something that makes sense spatially for it.
J.Doe
CC BY-SA 4.0
2018-09-06T04:10:09.987Z
@J.Doe Are you saying that the triangles end up in different strips (and thus in different draw calls) between the two configurations In that case, I'm confused by the statement in your original question stating that "in memory a single strips triangles are very spread out" (this is a comment about memory configuration 1).
John Calsbeek
CC BY-SA 4.0
2018-09-06T16:53:38.387Z
1 draw call for all of the triangles
J.Doe
CC BY-SA 4.0
2018-09-07T16:41:20.713Z
@J.Doe - I don't quite understand your statements when you say "concentrated chunk of memory is updated" in memory config 1 and "writing is more spread out" in config 2. However I think what John is trying to tell you, for memory config 2, the data for the next triangle that needs to be drawn is pulled in when the GPU accesses the data for the first one. This is because GPU accesses memory in blocks. So that whole block gets written into the cache. For the next triangle it can fetch directly from high speed cache instead of that slow access from memory, hence the speed up.
gallickgunner
CC BY-SA 4.0
2018-09-04T01:20:59.480Z
2018-09-04T01:20:59.480Z
John Calsbeek
<p>So I have a bit of a predicament I am trying to solve. I have 10,000 triangle strips I am trying to draw. Each one has a z-order for the entire strip and within each strip the most recent triangles need to have a higher z order than the older one. So essentially the z order of a triangle is <code>(stripIndex / 10000) + (age / MAX_AGE)</code> where MAX_AGE is somewhere around 100 frames. I am not sure about this number yet.</p>

<p>At start time all of the data to store this is initialized and a ring buffer of some form is used so that the most recent triangle's are in a new position each frame.</p>

<p>This ring buffer is useful because only 2 triangles will have to be written for each frame per triangle strip so no data has to shift or move. You just over-write the oldest data.</p>

<p>The trick is I want to do alpha blending between these triangle strips so I need to have the geometry drawn in order from front to back. I am experimenting with different layouts that will allow me to minimize draw calls and CPU stress required. This is an unsolved problem for me but I think I am going to have to do 10001 draw calls. But that is not what this question is about.</p>

<p>So while trying to do this I ran into an odd GPU phenomenon I cant explain. Right now I am rendering with a simple no-depth additive blending Metal pipeline that looks like this.</p>

<pre><code>id&lt;MTLFunction&gt; vertex_f = [MetalCore compileShader:vertex];
id&lt;MTLFunction&gt; fragment_f = [MetalCore compileShader:fragment];

MTLRenderPipelineDescriptor *descriptor = [[MTLRenderPipelineDescriptor alloc] init];
descriptor.sampleCount = 1;
descriptor.fragmentFunction = fragment_f;
descriptor.vertexFunction = vertex_f;
descriptor.colorAttachments[0].pixelFormat = MTLPixelFormatBGRA8Unorm_sRGB;
descriptor.colorAttachments[0].blendingEnabled = YES;
descriptor.colorAttachments[0].rgbBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].alphaBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].sourceRGBBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].sourceAlphaBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].destinationRGBBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].destinationAlphaBlendFactor = MTLBlendFactorOne;
</code></pre>

<p>As I said at the beginning I create a buffer that holds all of the triangles necessary to draw each of the strips. The buffer is the same size every time and the triangle count is also constant. The buffers do have indices and that is also constant and created at the beginning. The underlying simulation that controls these triangle strips is also constant. However under two different memory layout conditions there are significantly different frame rates even under multiple trials. I cant figure out why. </p>

<p>From now on I will refer to 10000 as MAX_STRIPS</p>

<p>Memory configuration 1 has a triangle for the n'th strip in the i'th ring buffer position located in the triangle buffer at <code>i*MAX_STRIPS*4 + n*4</code> th position in the buffer. So in memory a single strips triangles are very spread out but a concentrated chunk of memory is updated each frame. </p>

<p>Memory configuration 2 has a triangle for the n'th strip in the i'th ring buffer position located in the triangle buffer at the <code>n*MAX_AGE*4 + i*4</code> th position in the buffer. All of a single triangle strips vertices are close together in memory however writing is more spread out.</p>

<p>For whatever reason configuration 1 takes significantly longer than configuration 2 to render.</p>

<p>Now I could understand a small difference in CPU time but what I dont get is this difference in GPU time. Any clue as to why this might be happening.</p>

<p>The following compares debugging information of these two methods. The left side is memory configuration 1. The right is memory configuration 2. Big difference are highlighted.
<a href="https://i.sstatic.net/OFdDU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/OFdDU.png" alt="enter image description here"></a></p>

<p>Here is the vertex shader for all interested </p>

<pre><code>struct VertexOutAlpha {
    float4 position [[ position ]];
    float4 color;
};
struct StripVertex {
    float3 data; //z value is the z value for the entire strip. Triangles will have different z positions
    float2 color;//x = color, y = time
};
vertex VertexOutAlpha strip_vertex(constant StripVertex * vertexBuffer [[ buffer(0) ]],
                              indexType vid [[ vertex_id ]],
                              constant matrix_float3x3* matrix [[buffer(1)]],
                              constant float* gameTime [[buffer(2)]]) {
    StripVertex vert = vertexBuffer[vid];
    VertexOutAlpha out;
    float dimmer = 1.0 - clamp(((*gameTime - 0.016) - vert.color.y) * 10.0, 0.0, 1.0);
    const float levelSize = 1.0 / (MAX_STRIPS + 0.01);
    out.position = float4((*matrix * float3(vert.data.x, vert.data.y, 1.0)).xy, vert.data.z + (1 - dimmer) * levelSize, 1.0);

     out.color = float4(HSVtoRGB(vert.color.x, 1.0, dimmer), 1.0);
    return out;
}
</code></pre>

<p>And the generation of the vertex buffer for all interested</p>

<pre><code>    //triangleStripDataPtr is pre-allocated and alligned to the page size
    triangleStripDataBuffer = [self.device newBufferWithBytesNoCopy:triangleStripDataPtr length:4 * MAX_STRIPS *  MAX_AGE options:MTLResourceStorageModeShared deallocator:nil];
    assert(triangleStripDataBuffer != nil);
</code></pre>

<p>Does anyone have a good explanation for why this is happening?</p>

CC BY-SA 4.0
2018-09-03T15:22:14.383Z
You say that configuration 2 takes longer to render, but in your profiling screenshots, configuration 2 takes 55ms instead of 81ms. Is the sentence starting with "For some reason" backward?
John Calsbeek
CC BY-SA 4.0
2018-09-03T19:52:01.447Z
It is. Edited question
J.Doe
<gpu><vertex-buffer-object><metal>
Same draw call causing extremely different GPU usage under different triangle orders
J.Doe
# Same draw call causing extremely different GPU usage under different triangle orders

(Asked by: username_0 on 2018-08-30)

So I have a bit of a predicament I am trying to solve. I have 10,000 triangle strips I am trying to draw. Each one has a z-order for the entire strip and within each strip the most recent triangles need to have a higher z order than the older one. So essentially the z order of a triangle is `(stripIndex / 10000) + (age / MAX_AGE)` where MAX\_AGE is somewhere around 100 frames. I am not sure about this number yet.

At start time all of the data to store this is initialized and a ring buffer of some form is used so that the most recent triangle's are in a new position each frame.

This ring buffer is useful because only 2 triangles will have to be written for each frame per triangle strip so no data has to shift or move. You just over-write the oldest data.

The trick is I want to do alpha blending between these triangle strips so I need to have the geometry drawn in order from front to back. I am experimenting with different layouts that will allow me to minimize draw calls and CPU stress required. This is an unsolved problem for me but I think I am going to have to do 10001 draw calls. But that is not what this question is about.

So while trying to do this I ran into an odd GPU phenomenon I cant explain. Right now I am rendering with a simple no-depth additive blending Metal pipeline that looks like this.

```
id<MTLFunction> vertex_f = [MetalCore compileShader:vertex];
id<MTLFunction> fragment_f = [MetalCore compileShader:fragment];

MTLRenderPipelineDescriptor *descriptor = [[MTLRenderPipelineDescriptor alloc] init];
descriptor.sampleCount = 1;
descriptor.fragmentFunction = fragment_f;
descriptor.vertexFunction = vertex_f;
descriptor.colorAttachments[0].pixelFormat = MTLPixelFormatBGRA8Unorm_sRGB;
descriptor.colorAttachments[0].blendingEnabled = YES;
descriptor.colorAttachments[0].rgbBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].alphaBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].sourceRGBBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].sourceAlphaBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].destinationRGBBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].destinationAlphaBlendFactor = MTLBlendFactorOne;

```

As I said at the beginning I create a buffer that holds all of the triangles necessary to draw each of the strips. The buffer is the same size every time and the triangle count is also constant. The buffers do have indices and that is also constant and created at the beginning. The underlying simulation that controls these triangle strips is also constant. However under two different memory layout conditions there are significantly different frame rates even under multiple trials. I cant figure out why.

From now on I will refer to 10000 as MAX\_STRIPS

Memory configuration 1 has a triangle for the n'th strip in the i'th ring buffer position located in the triangle buffer at `i*MAX_STRIPS*4 + n*4` th position in the buffer. So in memory a single strips triangles are very spread out but a concentrated chunk of memory is updated each frame.

Memory configuration 2 has a triangle for the n'th strip in the i'th ring buffer position located in the triangle buffer at the `n*MAX_AGE*4 + i*4` th position in the buffer. All of a single triangle strips vertices are close together in memory however writing is more spread out.

For whatever reason configuration 1 takes significantly longer than configuration 2 to render.

Now I could understand a small difference in CPU time but what I dont get is this difference in GPU time. Any clue as to why this might be happening.

The following compares debugging information of these two methods. The left side is memory configuration 1. The right is memory configuration 2. Big difference are highlighted.
[![enter image description here](https://i.sstatic.net/OFdDU.png)](https://i.sstatic.net/OFdDU.png)

Here is the vertex shader for all interested

```
struct VertexOutAlpha {
    float4 position [[ position ]];
    float4 color;
};
struct StripVertex {
    float3 data; //z value is the z value for the entire strip. Triangles will have different z positions
    float2 color;//x = color, y = time
};
vertex VertexOutAlpha strip_vertex(constant StripVertex * vertexBuffer [[ buffer(0) ]],
                              indexType vid [[ vertex_id ]],
                              constant matrix_float3x3* matrix [[buffer(1)]],
                              constant float* gameTime [[buffer(2)]]) {
    StripVertex vert = vertexBuffer[vid];
    VertexOutAlpha out;
    float dimmer = 1.0 - clamp(((*gameTime - 0.016) - vert.color.y) * 10.0, 0.0, 1.0);
    const float levelSize = 1.0 / (MAX_STRIPS + 0.01);
    out.position = float4((*matrix * float3(vert.data.x, vert.data.y, 1.0)).xy, vert.data.z + (1 - dimmer) * levelSize, 1.0);

     out.color = float4(HSVtoRGB(vert.color.x, 1.0, dimmer), 1.0);
    return out;
}

```

And the generation of the vertex buffer for all interested

```
    //triangleStripDataPtr is pre-allocated and alligned to the page size
    triangleStripDataBuffer = [self.device newBufferWithBytesNoCopy:triangleStripDataPtr length:4 * MAX_STRIPS *  MAX_AGE options:MTLResourceStorageModeShared deallocator:nil];
    assert(triangleStripDataBuffer != nil);

```

Does anyone have a good explanation for why this is happening?

**Tags:** gpu, vertex-buffer-object, metal

> **Comments:**
> **username_1**: You say that configuration 2 takes longer to render, but in your profiling screenshots, configuration 2 takes 55ms instead of 81ms. Is the sentence starting with "For some reason" backward?  
> **username_0**: It is. Edited question  

---

## 1 answers

### Answer by username_1 on 2018-09-04. Score: 2

I think you're seeing cache effects. The GPU will begin vertex processing for each strip in sequential order, and in configuration 1 the GPU will fetch a different chunk of memory for each triangle in the strip (since, as you said, the strips themselves are not contiguous in memory in this configuration).

If I understand correctly, each entry in the ring buffer you are referring to is a triangle, so you will have at least 3 vertices adjacent to each other (in configuration 1). This has decent [spatial locality](https://en.wikipedia.org/wiki/Locality_of_reference), but fetching this memory will probably also pull in memory for adjacent vertices, which won't be immediately useful (since it belongs to a different strip).

In configuration 2, the adjacent vertices *will* be immediately useful, since they're the next vertices the GPU will need to render for the current triangle strip. The result is, probably, less total memory traffic.

This is a somewhat simplified explanation (for example, it brushes over the parallelism occurring on the GPU), but it seems to match what you're observing. One way to justify the choice of configuration 2 over configuration 1 is that you're making it easier on the GPU at the cost of the CPU—however, the CPU has 1/MAX\_AGE of the work to do as the GPU each frame, so optimizing for CPU updating ("a concentrated chunk of memory is updated each frame") will be less valuable.

> **Comments:**
> **username_0**: Why would it fetch things in a non sequential order? The index buffer has it go linearly through the structure. So the draw call and index buffer are the same in both scenarios. Sorry if that wasnt clear in the original question. The only way I could see this mattering is if the gpu overrides my index buffer to something that makes sense spatially for it.  
> **username_1**: @username_0 Are you saying that the triangles end up in different strips (and thus in different draw calls) between the two configurations In that case, I'm confused by the statement in your original question stating that "in memory a single strips triangles are very spread out" (this is a comment about memory configuration 1).  
> **username_0**: 1 draw call for all of the triangles  
> **username_2**: @username_0 - I don't quite understand your statements when you say "concentrated chunk of memory is updated" in memory config 1 and "writing is more spread out" in config 2. However I think what username_1 is trying to tell you, for memory config 2, the data for the next triangle that needs to be drawn is pulled in when the GPU accesses the data for the first one. This is because GPU accesses memory in blocks. So that whole block gets written into the cache. For the next triangle it can fetch directly from high speed cache instead of that slow access from memory, hence the speed up.  

---

