<p>So I've written a GPU based <a href="https://github.com/gallickgunner/Yune" rel="nofollow noreferrer">pathtracer</a> using OpenCL-GL interoperability. The system uses the Mega-Kernel Approach instead of a wavefront one as I was aiming it as an educational software for other young people who wanted to dive into raytracing/pathtracing but didn't want to mess with all the boilerplate code (opening windows, getting opengl context, etc) similar to coding it on shadertoy but with the added feature of loading models, having other benchmarking facilities etc.</p>

<p>I basically spawn as many work-items as there are pixels. Then each work-item shoots a single ray through that pixel and calculates the color. Currently on my GTX 1660, rendering around 5-6k Triangles give only 5-10 fps in 1280x720 resolution. I suppose its fairly slow even though I implemented a SAH based BVH and Möller–Trumbore ray-triangle intersection. Hence one big problem is that since the kernel takes too much time, loading big models causes a timeout error. I've already extended the timeout delay through the registery. In order to avoid it, I've also currently broken the image into 4 blocks and passed 4 mega kernels to do the job. This reduces the time of a single kernel, nevertheless the problem still remains.</p>

<p>I've looked into the <a href="https://www.researchgate.net/publication/258395638_The_Brigade_Renderer_A_Path_Tracer_for_Real-Time_Games" rel="nofollow noreferrer">paper</a> for Brigade renderer which is a realtime GPU pathtracer but couldn't understand what approach they followed. According to the text, they are talking about a single kernel per pixel,</p>

<blockquote>
  <p>The path tracing algorithm can be efficiently implemented as on the
  GPU, using a single kernel per pixel. The kernel loops over the
  samples for a pixel and outputs the final color. This limits memory
  access to read-only scene access and a single write for the final
  pixel color.</p>
</blockquote>

<ol>
<li><p>However the looping part is strange. If there is supposed to be a single kernel launched for every pixel, then the global work items or the threads must be the samples for that pixel. Why are they looping over the samples inside the kernel? </p></li>
<li><p>Secondly, I've also found <a href="https://jacco.ompf2.com/2019/07/18/wavefront-path-tracing/" rel="nofollow noreferrer">sources</a> that suggest that Brigade, infact used the wavefront approach?</p></li>
</ol>

<p>I also realize the old Brigade demos are done with either a single or dual Titan X but the difference isn't that great. I admit I haven't taken cache coherency and such things into account but these are hard to do with the mega-kernel approach anyway. <strong>So in short the main crux of what I want to know is, Is the megakernel approach bound to give this FPS OR Am I doing something wrong here? Any tips on how I can improve the FPS for my mega-kernel pathtracer would be appreciated.</strong> Link to the <a href="https://github.com/gallickgunner/Yune/blob/master/kernels/legacy/udpt.cl" rel="nofollow noreferrer">kernel</a>.</p>

CC BY-SA 4.0
2020-03-23T07:20:04.023Z
I always got the impression divergence is a bottleneck here. How are you handling bounces ? I found it was quicker to update a ray-state structure instead of updating the accumulation buffer (which requires a fully resolved path) so each PT pass updates one step of each per pixel ray. This way all threads are doing more even workloads. This complicates image output as pixels can only be output when a ray has completed, but it does mean a lot of resources are not idly waiting on that 1 path that needed a full 8 or whatever bounces. This is not a solution for your problem,more a talking point
PaulHK
CC BY-SA 4.0
2020-03-25T06:16:31.343Z
Well since it's a megakernel, single pass approach, Each work item is assigned to one pixel and the path terminates through Russian Roulette. I realize this can increase  the divergence but I have no other choice. I guess there really aren't that many ways to improve mega-kernel performance.
gallickgunner
CC BY-SA 4.0
2020-03-25T09:41:49.537Z
How about occupancy ? What happens to neighbouring paths when 1 path is at worst case maximum bounces ?
PaulHK
CC BY-SA 4.0
2020-03-25T15:01:35.743Z
That's the thing, since all the work items in a workgroup are operating on separate pixels, the whole workgroup will get delayed till whatever work-item has completed the maximum bounces
gallickgunner
<rendering><pathtracing><gpgpu><opencl>
How to achieve higher FPS for my GPU pathtracer?
gallickgunner
# How to achieve higher FPS for my GPU pathtracer?

(Asked by: username_0 on 2020-03-17)

So I've written a GPU based [pathtracer](https://github.com/username_0/Yune) using OpenCL-GL interoperability. The system uses the Mega-Kernel Approach instead of a wavefront one as I was aiming it as an educational software for other young people who wanted to dive into raytracing/pathtracing but didn't want to mess with all the boilerplate code (opening windows, getting opengl context, etc) similar to coding it on shadertoy but with the added feature of loading models, having other benchmarking facilities etc.

I basically spawn as many work-items as there are pixels. Then each work-item shoots a single ray through that pixel and calculates the color. Currently on my GTX 1660, rendering around 5-6k Triangles give only 5-10 fps in 1280x720 resolution. I suppose its fairly slow even though I implemented a SAH based BVH and Möller–Trumbore ray-triangle intersection. Hence one big problem is that since the kernel takes too much time, loading big models causes a timeout error. I've already extended the timeout delay through the registery. In order to avoid it, I've also currently broken the image into 4 blocks and passed 4 mega kernels to do the job. This reduces the time of a single kernel, nevertheless the problem still remains.

I've looked into the [paper](https://www.researchgate.net/publication/258395638_The_Brigade_Renderer_A_Path_Tracer_for_Real-Time_Games) for Brigade renderer which is a realtime GPU pathtracer but couldn't understand what approach they followed. According to the text, they are talking about a single kernel per pixel,

> The path tracing algorithm can be efficiently implemented as on the
> GPU, using a single kernel per pixel. The kernel loops over the
> samples for a pixel and outputs the final color. This limits memory
> access to read-only scene access and a single write for the final
> pixel color.

1. However the looping part is strange. If there is supposed to be a single kernel launched for every pixel, then the global work items or the threads must be the samples for that pixel. Why are they looping over the samples inside the kernel?
2. Secondly, I've also found [sources](https://jacco.ompf2.com/2019/07/18/wavefront-path-tracing/) that suggest that Brigade, infact used the wavefront approach?

I also realize the old Brigade demos are done with either a single or dual Titan X but the difference isn't that great. I admit I haven't taken cache coherency and such things into account but these are hard to do with the mega-kernel approach anyway. **So in short the main crux of what I want to know is, Is the megakernel approach bound to give this FPS OR Am I doing something wrong here? Any tips on how I can improve the FPS for my mega-kernel pathtracer would be appreciated.** Link to the [kernel](https://github.com/username_0/Yune/blob/master/kernels/legacy/udpt.cl).

**Tags:** rendering, pathtracing, gpgpu, opencl

> **Comments:**
> **username_1**: I always got the impression divergence is a bottleneck here. How are you handling bounces ? I found it was quicker to update a ray-state structure instead of updating the accumulation buffer (which requires a fully resolved path) so each PT pass updates one step of each per pixel ray. This way all threads are doing more even workloads. This complicates image output as pixels can only be output when a ray has completed, but it does mean a lot of resources are not idly waiting on that 1 path that needed a full 8 or whatever bounces. This is not a solution for your problem,more a talking point  
> **username_0**: Well since it's a megakernel, single pass approach, Each work item is assigned to one pixel and the path terminates through Russian Roulette. I realize this can increase  the divergence but I have no other choice. I guess there really aren't that many ways to improve mega-kernel performance.  
> **username_1**: How about occupancy ? What happens to neighbouring paths when 1 path is at worst case maximum bounces ?  
> **username_0**: That's the thing, since all the work items in a workgroup are operating on separate pixels, the whole workgroup will get delayed till whatever work-item has completed the maximum bounces  

---

## 0 answers

