<blockquote>
<p>Is it possible to trigger OpenGL to render a new image consisting of just one pixel for the refracted ray in the main function in the fragment (pixel?) shader?</p>
</blockquote>
<p>OpenGL does not know what your data is. It only knows what you tell it. It knows that you've bound some buffers and submitted some commands to render with them. It doesn't keep track of that information after it has processed those commands.</p>
<p>In short, it doesn't know what your scene is. So even if a fragment shader could provoke rendering (which it cannot), it wouldn't know <em>what</em> to render.</p>
<blockquote>
<p>If it is not possible to trigger the rendering of a new, single-pixel image in the main function in the fragment shader in OpenGL, does Vulkan, which is supposed to be more powerful, offer this possibility?</p>
</blockquote>
<p>No.</p>
<p>What Vulkan has is an (optional) raytracing rendering pipeline. It involves building data structures that represent &quot;the scene&quot; in some way, as well as dealing with a bunch of different shader stages that can create, test, and manipulate rays.</p>
<p>This is not a simple system and it doesn't work at all like the standard rasterization graphics pipeline. It's also Vulkan (or D3D12) only, as the complexities of it are too great for OpenGL as an API to handle.</p>

CC BY-SA 4.0
2023-06-02T15:21:58.853Z
2023-06-02T15:21:58.853Z
Nicol Bolas
<p>I was looking at <a href="https://sketchfab.com/3d-models/tfa-f5a74cf164c54e4a943352d750b8fac2" rel="nofollow noreferrer">this glass ball rendering</a> with an image inside of it (zoom out to see the glass ball), and noticed that it looks like they simulate the refraction of light as it crosses from inside of the glass ball to the air outside. I know that you can achieve this either by rendering to a texture and then using that texture on the surface of the transparent object, or moving the vertices of objects inside/behind the transparent object, but both of those methods are problematic in my situation.</p>
<p>I want to simulate the transition between two spaces where light bends differently (If you will, you can think of it kind of like I'm entering a <a href="https://www.youtube.com/watch?v=gQF-7yZMdnA" rel="nofollow noreferrer">worm</a> <a href="https://www.youtube.com/watch?v=FR2TE_Wf5fw" rel="nofollow noreferrer">hole</a> and ending up at another location), and you should be able to smoothly transition between the two spaces without being able to see the transition. For the rendering, the boundary between the two spaces can simply be treated as a refracting surface, and I would like to use ray tracing to achieve this refraction, where I only trace a single refracting ray and no reflecting ray. For this, I have some questions:</p>
<ol>
<li><p>Is it possible to trigger OpenGL to render a new image consisting of just one pixel for the refracted ray in the <code>main</code> function in the fragment (pixel?) shader? This would allow me to rely on a lot of functionality that already exists in OpenGL which I imagine that I would otherwise have have to implement myself, such as testing intersections between the ray and polygons in the scene, and interpolation of vertex values to the point on the polygon that was hit. If I could trigger OpenGL to render a new image, this would also make it easier to get a more consistently rendered image, as I want to avoid getting an image that changes discontinuously when I transition from being on one side of the boundary to being on the other side of the boundary.</p>
</li>
<li><p>If it is not possible to trigger the rendering of a new, single-pixel image in the <code>main</code> function in the fragment shader in OpenGL, does Vulkan, which is supposed to be more powerful, offer this possibility?</p>
</li>
<li><p>If I can't trigger OpenGL or Vulkan to render a new image in the <code>main</code> function in the fragment shader, do I have to manually calculate which polygon the refracted ray hits and where on it it hits, or can I still get OpenGL/Vulkan to do that automatically for me somehow? If I have to calculate this manually, how can I do that most easily? Does OpenGL/Vulkan provide me with some functions that can make this process easier for me?</p>
</li>
<li><p>Also, for the refracted ray, can I make OpenGL/Vulkan call the <code>main</code> function in the fragment shader with the appropriate interpolated values for all variables (like it does normally) for each successive hit, or do I have to interpolate those manually somehow too? Again, does OpenGL/Vulkan provide me with some functions that can make this process easier for me if I have to do it manually?</p>
</li>
</ol>

CC BY-SA 4.0
2023-06-01T06:05:32.150Z
I'd like to answer your question, but there are a few issues that I don't understand. - Light bending differently? Will it still obey [Snell's law](https://en.wikipedia.org/wiki/Snell%27s_law)? but with different refractive indices? -- this could be achieved by altering properties of the material of the sphere - A fragment shader cannot move a pixel, a vertex in a vertex shader might be more useful, this would be rasterised and cause pixel(s) to appear for the fragment shader to process, and display - Vulkan does have extensions specifically for ray-tracing, but I have not explored this - It m
Nick
CC BY-SA 4.0
2023-06-28T15:55:36.187Z
@YvesDaoust How would you render the glass ball (or any of the worm holes) I linked to "normally"?
HelloGoodbye
<opengl><raytracing><shader><vulkan><refraction>
Raytracing in OpenGL or Vulkan?
HelloGoodbye
# Raytracing in OpenGL or Vulkan?

(Asked by: username_0 on 2023-03-03)

I was looking at [this glass ball rendering](https://sketchfab.com/3d-models/tfa-f5a74cf164c54e4a943352d750b8fac2) with an image inside of it (zoom out to see the glass ball), and noticed that it looks like they simulate the refraction of light as it crosses from inside of the glass ball to the air outside. I know that you can achieve this either by rendering to a texture and then using that texture on the surface of the transparent object, or moving the vertices of objects inside/behind the transparent object, but both of those methods are problematic in my situation.

I want to simulate the transition between two spaces where light bends differently (If you will, you can think of it kind of like I'm entering a [worm](https://www.youtube.com/watch?v=gQF-7yZMdnA) [hole](https://www.youtube.com/watch?v=FR2TE_Wf5fw) and ending up at another location), and you should be able to smoothly transition between the two spaces without being able to see the transition. For the rendering, the boundary between the two spaces can simply be treated as a refracting surface, and I would like to use ray tracing to achieve this refraction, where I only trace a single refracting ray and no reflecting ray. For this, I have some questions:

1. Is it possible to trigger OpenGL to render a new image consisting of just one pixel for the refracted ray in the `main` function in the fragment (pixel?) shader? This would allow me to rely on a lot of functionality that already exists in OpenGL which I imagine that I would otherwise have have to implement myself, such as testing intersections between the ray and polygons in the scene, and interpolation of vertex values to the point on the polygon that was hit. If I could trigger OpenGL to render a new image, this would also make it easier to get a more consistently rendered image, as I want to avoid getting an image that changes discontinuously when I transition from being on one side of the boundary to being on the other side of the boundary.
2. If it is not possible to trigger the rendering of a new, single-pixel image in the `main` function in the fragment shader in OpenGL, does Vulkan, which is supposed to be more powerful, offer this possibility?
3. If I can't trigger OpenGL or Vulkan to render a new image in the `main` function in the fragment shader, do I have to manually calculate which polygon the refracted ray hits and where on it it hits, or can I still get OpenGL/Vulkan to do that automatically for me somehow? If I have to calculate this manually, how can I do that most easily? Does OpenGL/Vulkan provide me with some functions that can make this process easier for me?
4. Also, for the refracted ray, can I make OpenGL/Vulkan call the `main` function in the fragment shader with the appropriate interpolated values for all variables (like it does normally) for each successive hit, or do I have to interpolate those manually somehow too? Again, does OpenGL/Vulkan provide me with some functions that can make this process easier for me if I have to do it manually?

**Tags:** opengl, raytracing, shader, vulkan, refraction

> **Comments:**
> **username_1**: I'd like to answer your question, but there are a few issues that I don't understand. - Light bending differently? Will it still obey [Snell's law](https://en.wikipedia.org/wiki/Snell%27s_law)? but with different refractive indices? -- this could be achieved by altering properties of the material of the sphere - A fragment shader cannot move a pixel, a vertex in a vertex shader might be more useful, this would be rasterised and cause pixel(s) to appear for the fragment shader to process, and display - Vulkan does have extensions specifically for ray-tracing, but I have not explored this - It m  
> **username_0**: @YvesDaoust How would you render the glass ball (or any of the worm holes) I linked to "normally"?  

---

## 1 answers

### Answer by username_2 on 2023-06-02. Score: 1

> Is it possible to trigger OpenGL to render a new image consisting of just one pixel for the refracted ray in the main function in the fragment (pixel?) shader?

OpenGL does not know what your data is. It only knows what you tell it. It knows that you've bound some buffers and submitted some commands to render with them. It doesn't keep track of that information after it has processed those commands.

In short, it doesn't know what your scene is. So even if a fragment shader could provoke rendering (which it cannot), it wouldn't know *what* to render.

> If it is not possible to trigger the rendering of a new, single-pixel image in the main function in the fragment shader in OpenGL, does Vulkan, which is supposed to be more powerful, offer this possibility?

No.

What Vulkan has is an (optional) raytracing rendering pipeline. It involves building data structures that represent "the scene" in some way, as well as dealing with a bunch of different shader stages that can create, test, and manipulate rays.

This is not a simple system and it doesn't work at all like the standard rasterization graphics pipeline. It's also Vulkan (or D3D12) only, as the complexities of it are too great for OpenGL as an API to handle.


---

