<p>While the existing answer adress your question quite well, I'd like to add some general advice, even if you might already be aware of that. Rather than just <em>wasting</em> that fourth component on the GPU, you might as well <strong>use it for additional data</strong>.</p>

<p>It might be an obvious suggestion, but a 4th unused 32-bit word can store a variety of useful data:</p>

<ul>
<li>8-bit RGBA colour components.</li>
<li>2D texture coordinates in either 16-bit half-precision floating point or normalized fixed-point format, depending on your precision and range requirements.</li>
<li>packed normal data using some of the excellent normal compression formats discussed in <a href="http://jcgt.org/published/0003/02/01/" rel="nofollow noreferrer">this survey of unit vector representations</a>.</li>
</ul>

<p>This would let you use packed 128-bit SSE copy instructions for uploading data (especially since mapped buffer memory is usally well aligned) after just a bit of (easily SSE-optimizable) packing. Of course the viablity of this highly depends on your data. But if you have e.g. 5 floats for position and texCoord or 6 for position and normal, trying one of these schemes might be a neat consideration.</p>

<p>And using 4-component vectors for your vertices on the GPU can have additional advantages beyond just SSEability on the CPU.</p>

<ul>
<li>It <em>might</em> help your <strong>GPU optimize on-chip memory reads/writes</strong> if your data is well aligned (although, of course it has to be measured individually if there's a practical advantage).</li>
<li>But even disregarding performance it can make development a little more convenient, too. Speaking from OpenGL experience, <a href="https://computergraphics.stackexchange.com/q/5810/6">using 3-component vertex data in shader storage buffers (i.e. for advanced memory access and/or compute shaders) is a bit of a hassle</a> because of GLSL's data layout. A buffer filled with <strong>4-component vectors makes for generally cleaner and more streamlined code</strong>.</li>
</ul>

CC BY-SA 4.0
2018-07-03T05:32:56.410Z
Yeah, vec3 and std140 don't play nice at all! Anything I've done with compute shaders I just used vec4 and try to pack something else into the w if possible.
russ
CC BY-SA 4.0
2018-06-25T13:05:26.983Z
2018-06-25T13:05:26.983Z
Christian Rau
<p>The conversion from option A is likely faster than you think. See, for example, <a href="https://software.intel.com/en-us/articles/3d-vector-normalization-using-256-bit-intel-advanced-vector-extensions-intel-avx" rel="nofollow noreferrer">Intel's version</a> which claims to do the round trip at 1.5 cycles per vector with AVX.</p>

<p>There are also some operations which don't require the conversion at all. For example, if you want to compute a bounding box, you can do it directly with the packed data in chunks of 4 vectors and fix things up once you've gone through the entire array.</p>

<p>And finally a word of warning: a vector class as presented in the article is not how you write high performance code nor how you generally should use SIMD.</p>

CC BY-SA 4.0
2018-07-01T23:10:45.130Z
2018-07-01T23:10:45.130Z
Olivier
<p>There's a third option you've neglected. Because of how graphics APIs work, typically you have to copy the data between the CPU and GPU anyway - even if you're on a mobile SoC where they share the same physical memory. This copy is the ideal time to take the hit of changing format, since you have to touch all of the memory anyway. So the copy in user memory, owned by the CPU, can be stored padded for SIMD alignment. Then, when you need to send that data to GPU memory, instead of doing a memcpy, just read in the 4-vectors and write out 3-vectors. It's less memory bandwidth than a normal copy (you're reading just as much, but writing less) and negligible CPU work.</p>

<p>If you've got that fourth float doing nothing, you might as well use it and make your co-ordinates homogeneous instead of just xyz. Depending on what compute you're doing on the CPU, that might make some things easier (transforms are the obvious example, but also reducing divisions if you were renormalising your normals a lot before).</p>

<p>This is similar but not identical to a trick I used when implementing OpenGL texture upload. On ARM, most Neon instructions operate at memory bandwidth rate, so if you're already doing a memcpy, it's free to do a pipeline-worth of instructions at the same time. You can use those free instructions to do pixel format conversion <em>and</em> reordering to Z-order or U-order.</p>

CC BY-SA 3.0
2017-09-28T07:48:35.363Z
Can you elaborate on this a bit? As far as i can tell, the GL methods for loading data to buffers (glBufferData and glBufferSubData) just copy a contiguous block of bytes across. Is it possible to somehow get a pointer into GPU memory and write your own copy routine?
russ
CC BY-SA 3.0
2017-09-28T08:54:21.313Z
Ahh of course, you can map the buffer then copy directly. Derp :-/
russ
CC BY-SA 3.0
2017-09-28T07:24:17.870Z
2017-09-28T07:24:17.870Z
Dan Hulme
<p>I've been getting interested in SIMD programming on CPU using SSE recently, but I'm a complete noob on the subject. I found <a href="http://www.codersnotes.com/notes/maths-lib-2016/" rel="nofollow noreferrer">this article</a> describing how to make an efficient float3 type using the recent __vectorcall convention, and it made me wonder how best to deal with the wasted space inherent in using a 128-bit type to represent vectors smaller than 4.</p>

<p>For a typical use case, you might want to transform a bunch of positions and normals on the CPU in order to do a batched draw call. When transmitting the data over to the GPU, though, ideally you'd want this as tightly packed arrays of float3, either interleaved or in separate buffers.</p>

<p>It seems like there are two options here:</p>

<p>A - store the data tightly packed, and unpack and repack to and from SSE-friendly format when you need to manipulate it, taking the CPU performance hit, or</p>

<p>B - store the data loosely packed on 16-byte boundaries, SSE it directly, and take the bandwidth and VRAM hit when sending to GPU.</p>

<p>Which of these options would be preferred in a real-world 3D engine, or under what circumstances would you prefer one to the other?   </p>

<performance><memory><data-structure>
Dealing with wasted space when using SIMD on CPU for vector maths
russ
# Dealing with wasted space when using SIMD on CPU for vector maths

(Asked by: username_0 on 2017-09-28)

I've been getting interested in SIMD programming on CPU using SSE recently, but I'm a complete noob on the subject. I found [this article](http://www.codersnotes.com/notes/maths-lib-2016/) describing how to make an efficient float3 type using the recent \_\_vectorcall convention, and it made me wonder how best to deal with the wasted space inherent in using a 128-bit type to represent vectors smaller than 4.

For a typical use case, you might want to transform a bunch of positions and normals on the CPU in order to do a batched draw call. When transmitting the data over to the GPU, though, ideally you'd want this as tightly packed arrays of float3, either interleaved or in separate buffers.

It seems like there are two options here:

A - store the data tightly packed, and unpack and repack to and from SSE-friendly format when you need to manipulate it, taking the CPU performance hit, or

B - store the data loosely packed on 16-byte boundaries, SSE it directly, and take the bandwidth and VRAM hit when sending to GPU.

Which of these options would be preferred in a real-world 3D engine, or under what circumstances would you prefer one to the other?

**Tags:** performance, memory, data-structure


---

## 3 answers

### Answer by username_3 on 2017-09-28. Score: 6 | Accepted answer

There's a third option you've neglected. Because of how graphics APIs work, typically you have to copy the data between the CPU and GPU anyway - even if you're on a mobile SoC where they share the same physical memory. This copy is the ideal time to take the hit of changing format, since you have to touch all of the memory anyway. So the copy in user memory, owned by the CPU, can be stored padded for SIMD alignment. Then, when you need to send that data to GPU memory, instead of doing a memcpy, just read in the 4-vectors and write out 3-vectors. It's less memory bandwidth than a normal copy (you're reading just as much, but writing less) and negligible CPU work.

If you've got that fourth float doing nothing, you might as well use it and make your co-ordinates homogeneous instead of just xyz. Depending on what compute you're doing on the CPU, that might make some things easier (transforms are the obvious example, but also reducing divisions if you were renormalising your normals a lot before).

This is similar but not identical to a trick I used when implementing OpenGL texture upload. On ARM, most Neon instructions operate at memory bandwidth rate, so if you're already doing a memcpy, it's free to do a pipeline-worth of instructions at the same time. You can use those free instructions to do pixel format conversion *and* reordering to Z-order or U-order.

> **Comments:**
> **username_0**: Can you elaborate on this a bit? As far as i can tell, the GL methods for loading data to buffers (glBufferData and glBufferSubData) just copy a contiguous block of bytes across. Is it possible to somehow get a pointer into GPU memory and write your own copy routine?  
> **username_0**: Ahh of course, you can map the buffer then copy directly. Derp :-/  

---

### Answer by username_1 on 2018-06-25. Score: 3

While the existing answer adress your question quite well, I'd like to add some general advice, even if you might already be aware of that. Rather than just *wasting* that fourth component on the GPU, you might as well **use it for additional data**.

It might be an obvious suggestion, but a 4th unused 32-bit word can store a variety of useful data:

* 8-bit RGBA colour components.
* 2D texture coordinates in either 16-bit half-precision floating point or normalized fixed-point format, depending on your precision and range requirements.
* packed normal data using some of the excellent normal compression formats discussed in [this survey of unit vector representations](http://jcgt.org/published/0003/02/01/).

This would let you use packed 128-bit SSE copy instructions for uploading data (especially since mapped buffer memory is usally well aligned) after just a bit of (easily SSE-optimizable) packing. Of course the viablity of this highly depends on your data. But if you have e.g. 5 floats for position and texCoord or 6 for position and normal, trying one of these schemes might be a neat consideration.

And using 4-component vectors for your vertices on the GPU can have additional advantages beyond just SSEability on the CPU.

* It *might* help your **GPU optimize on-chip memory reads/writes** if your data is well aligned (although, of course it has to be measured individually if there's a practical advantage).
* But even disregarding performance it can make development a little more convenient, too. Speaking from OpenGL experience, [using 3-component vertex data in shader storage buffers (i.e. for advanced memory access and/or compute shaders) is a bit of a hassle](https://computergraphics.stackexchange.com/q/5810/6) because of GLSL's data layout. A buffer filled with **4-component vectors makes for generally cleaner and more streamlined code**.

> **Comments:**
> **username_0**: Yeah, vec3 and std140 don't play nice at all! Anything I've done with compute shaders I just used vec4 and try to pack something else into the w if possible.  

---

### Answer by username_2 on 2018-07-01. Score: 1

The conversion from option A is likely faster than you think. See, for example, [Intel's version](https://software.intel.com/en-us/articles/3d-vector-normalization-using-256-bit-intel-advanced-vector-extensions-intel-avx) which claims to do the round trip at 1.5 cycles per vector with AVX.

There are also some operations which don't require the conversion at all. For example, if you want to compute a bounding box, you can do it directly with the packed data in chunks of 4 vectors and fix things up once you've gone through the entire array.

And finally a word of warning: a vector class as presented in the article is not how you write high performance code nor how you generally should use SIMD.


---

