<p>There shouldn't be any performance penalty inherent in using a 3D group over a 2D one, the dimensions are just an aid for programmers to map invocations on to memory locations (since you can read or write wherever you like). </p>

<p>The group size affects which invocations can access the same block of groupshared memory, so it forces all warps from the same group to run on the same SMP. If you have a shader that needs to move stuff around a lot in its local memory block (e.g. a bitonic sorter), a large group size is preferable, but in your case it shouldn't make much difference. </p>

<p>I suspect your problem is with the idle threads within your warps. If I understand your question properly, you're using a thread per input texel or sample to do the texture reads, but then only a subset of these threads to do the processing? If so, you're sacrificing a lot of compute power within each warp. Remember, texture reads are done in minimum 2x2 blocks, and the results get cached in a way optimized for locality for the dimension of the texture (see <a href="https://computergraphics.stackexchange.com/questions/355/how-does-texture-cache-work-considering-multiple-shader-units">this question</a>). So although your general purpose shader might seem slow with all the texture reads, a lot of them will be effectively almost free. Also, the GPU does latency-hiding on slow memory reads, swapping in other warps until the result of the read is available. So your general purpose shader actually seems like the smart option here, as it maximizes compute power.</p>

CC BY-SA 3.0
2017-11-17T17:14:15.997Z
With regard to the idleness of threads: for MSAA $T$x, $T \times w \times h$ threads are used for retrieving data of which $w \times h$ threads perform the averaging and store the final results; for SSAA $T$x, $T \times T \times w \times h$ threads are used for retrieving data of which $w \times h$ threads perform the averaging and store the final results. Here, $w$ and $h$ represent the number of ouput texels for the width and height.
Matthias
CC BY-SA 3.0
2017-11-18T00:05:06.693Z
Agree with this answer. Using a thread per texture read isn't likely to get your data out of memory any faster than issuing $T$ texture reads per thread—in either case you're almost certainly going to be memory-bandwidth-bound. So by using all those extra threads you're just reducing your occupancy and taking longer to get through all the threads in the dispatch.
Nathan Reed
CC BY-SA 3.0
2017-11-17T15:45:29.037Z
2017-11-17T15:45:29.037Z
russ
<p>The maximum allowed number of threads per compute shader group is 1024 for Shader Model 5.0. Is it advisable with regard to performance to stay close to this maximum number?</p>

<p>In order to resolve SSAA and MSAA (down-scaling with appropriate tone mapping), I wrote some compute shaders.</p>

<p>One general compute shader works in all cases and performs one task without synchronization for each output texel and uses [numthreads(16, 16, 1)] (= 256 threads/group which is a multiple of NVidia's warp size of 32 and AMD's wavefront size of 64). Algorithmically, this does not result in the best performance, since for MSAA $T$x, each task performs $T$ texture loads from the radiance, normal and depth textures, and for SSAA $T$x, each task performs $T \times T$ texture loads from the radiance, normal and depth textures.</p>

<p>So instead of dispatching jobs in terms of output texels, I have some specialized compute shaders performing tasks with synchronization for each input texel (for SSAA) or for each subsample of an input texel (for MSAA). Here, each task performs $1$ texture load from the radiance, normal and depth textures and stores the results in group shared memory. Next, a <code>GroupMemoryBarrierWithGroupSync</code> takes place and only one job per output texel continues to finish the work. This is not ideal in case of many SSAA and MSAA samples, but for few samples such as MSAA 2x, one expects to see some gains. In practice, however, the general compute shader outperforms all the specialized compute shaders for both small and large numbers of samples/ouput texel?</p>

<p>SSAA 2x [numthreads(4, 16, 16)] = 1024 (multiple of 64): ~655 FPS vs ~577 FPS</p>

<p>SSAA 3x [numthreads(9, 8, 8)]   =  576 (multiple of 64): ~265 FPS vs ~200 FPS</p>

<p>SSAA 4x [numthreads(16, 8, 8)]  = 1024 (multiple of 64): ~127 FPS vs  ~75 FPS</p>

<p>Can this be solely due to dispatching more threads/group? Can this be due to the synchronization of all threads per group (close to the maximum allowed number of threads)? Or is there some performance one needs to sacrifice in case of using a 3D instead of 2D block of threads per group? </p>

CC BY-SA 3.0
2017-11-18T00:06:31.663Z
FYI it's generally considered more useful to report timings in milliseconds rather than in FPS, especially when comparing deltas between different techniques.
Nathan Reed
<hlsl><compute-shader>
Threads per compute shader group
Matthias
# Threads per compute shader group

(Asked by: username_0 on 2017-11-17)

The maximum allowed number of threads per compute shader group is 1024 for Shader Model 5.0. Is it advisable with regard to performance to stay close to this maximum number?

In order to resolve SSAA and MSAA (down-scaling with appropriate tone mapping), I wrote some compute shaders.

One general compute shader works in all cases and performs one task without synchronization for each output texel and uses [numthreads(16, 16, 1)] (= 256 threads/group which is a multiple of NVidia's warp size of 32 and AMD's wavefront size of 64). Algorithmically, this does not result in the best performance, since for MSAA $T$x, each task performs $T$ texture loads from the radiance, normal and depth textures, and for SSAA $T$x, each task performs $T \times T$ texture loads from the radiance, normal and depth textures.

So instead of dispatching jobs in terms of output texels, I have some specialized compute shaders performing tasks with synchronization for each input texel (for SSAA) or for each subsample of an input texel (for MSAA). Here, each task performs $1$ texture load from the radiance, normal and depth textures and stores the results in group shared memory. Next, a `GroupMemoryBarrierWithGroupSync` takes place and only one job per output texel continues to finish the work. This is not ideal in case of many SSAA and MSAA samples, but for few samples such as MSAA 2x, one expects to see some gains. In practice, however, the general compute shader outperforms all the specialized compute shaders for both small and large numbers of samples/ouput texel?

SSAA 2x [numthreads(4, 16, 16)] = 1024 (multiple of 64): ~655 FPS vs ~577 FPS

SSAA 3x [numthreads(9, 8, 8)] = 576 (multiple of 64): ~265 FPS vs ~200 FPS

SSAA 4x [numthreads(16, 8, 8)] = 1024 (multiple of 64): ~127 FPS vs ~75 FPS

Can this be solely due to dispatching more threads/group? Can this be due to the synchronization of all threads per group (close to the maximum allowed number of threads)? Or is there some performance one needs to sacrifice in case of using a 3D instead of 2D block of threads per group?

**Tags:** hlsl, compute-shader

> **Comments:**
> **username_1**: FYI it's generally considered more useful to report timings in milliseconds rather than in FPS, especially when comparing deltas between different techniques.  

---

## 1 answers

### Answer by username_2 on 2017-11-17. Score: 5 | Accepted answer

There shouldn't be any performance penalty inherent in using a 3D group over a 2D one, the dimensions are just an aid for programmers to map invocations on to memory locations (since you can read or write wherever you like).

The group size affects which invocations can access the same block of groupshared memory, so it forces all warps from the same group to run on the same SMP. If you have a shader that needs to move stuff around a lot in its local memory block (e.g. a bitonic sorter), a large group size is preferable, but in your case it shouldn't make much difference.

I suspect your problem is with the idle threads within your warps. If I understand your question properly, you're using a thread per input texel or sample to do the texture reads, but then only a subset of these threads to do the processing? If so, you're sacrificing a lot of compute power within each warp. Remember, texture reads are done in minimum 2x2 blocks, and the results get cached in a way optimized for locality for the dimension of the texture (see [this question](https://computergraphics.stackexchange.com/questions/355/how-does-texture-cache-work-considering-multiple-shader-units)). So although your general purpose shader might seem slow with all the texture reads, a lot of them will be effectively almost free. Also, the GPU does latency-hiding on slow memory reads, swapping in other warps until the result of the read is available. So your general purpose shader actually seems like the smart option here, as it maximizes compute power.

> **Comments:**
> **username_0**: With regard to the idleness of threads: for MSAA $T$x, $T \times w \times h$ threads are used for retrieving data of which $w \times h$ threads perform the averaging and store the final results; for SSAA $T$x, $T \times T \times w \times h$ threads are used for retrieving data of which $w \times h$ threads perform the averaging and store the final results. Here, $w$ and $h$ represent the number of ouput texels for the width and height.  
> **username_1**: Agree with this answer. Using a thread per texture read isn't likely to get your data out of memory any faster than issuing $T$ texture reads per thread—in either case you're almost certainly going to be memory-bandwidth-bound. So by using all those extra threads you're just reducing your occupancy and taking longer to get through all the threads in the dispatch.  

---

