<p>When cylinder and frustum intersect the hole will be a view into the internals of the cylinder. <strong>The hole is where external of the cylinder occlude frustum and internal of the cylinder does not occlude the frustum.</strong></p>

<p>Knowing this you should simply do the following to find the surface (or rather volume) made from the hole.</p>

<p>Render frustum without color and set depth accordingly. Set a stencil bit where the cylinder (external) occludes the frustum, without setting new depth. Clear the previously set bit where the internal of the cylinder occludes the frustum, without setting new depth. </p>

<p>Alternate between internal and external using CullFace.</p>

<p>The stencil mask you now have is the surface where the frustum hole is. Render the frustum (in color) masking it with the stencil. Make sure to use LEQUALS depth test to allow the frustum to render on top of itself. Voila. Note that the depth buffer holds the value of the unmasked frustum so If you need to render something within the hole you would have to render the whole scene again and mask it to only render in the hole unless depth test passes against the frustum. Left as student exercise.</p>

CC BY-SA 3.0
2018-04-10T14:45:28.950Z
Sorry for this late acknowledgement, but I had to shelve this project owing to other more pressing priorities, but now that I've some time to return to it, I'll investigate your suggestion. Thanks a ton!
Sharat V Chandrasekhar
CC BY-SA 3.0
2017-09-13T18:55:24.660Z
2017-09-13T18:55:24.660Z
Andreas
<p>I'm using OpenGL's stencil test to carve out a semicircular hole in a transition frustum with a cylinder oriented perpendicularly. However as one can see below, while this objective is achieved, the cylinder is still visible through the frustum which shouldn't be the case.</p>

<p><a href="https://i.sstatic.net/pmWBD.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/pmWBD.jpg" alt="enter image description here"></a>
<p></p>

<p>I'm using the following code fragment that I found at
<a href="https://en.wikibooks.org/wiki/OpenGL_Programming/Mini-Portal" rel="nofollow noreferrer">https://en.wikibooks.org/wiki/OpenGL_Programming/Mini-Portal</a></p>

<pre><code>glClear(GL_DEPTH_BUFFER_BIT);
glEnable(GL_STENCIL_TEST);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glDepthMask(GL_FALSE);
glStencilFunc(GL_NEVER, 1, 0xFF);
glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP);  // draw 1s on test fail (always)

// draw stencil pattern
glStencilMask(0xFF);
glClear(GL_STENCIL_BUFFER_BIT);  // needs mask=0xFF
DRAW_CYLINDER ();

glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glDepthMask(GL_TRUE);
glStencilMask(0x00);
// draw where stencil's value is 0
glStencilFunc(GL_EQUAL, 0, 0xFF);
/* (nothing to draw) */
// draw only where stencil's value is 1
glStencilFunc(GL_EQUAL, 1, 0xFF);

DRAW_FRUSTUM();

glDisable(GL_STENCIL_TEST);
</code></pre>

<p>Moreover I've found that the results are insensitive to the parameters of the <strong>glStencilOp</strong>  and <strong>DepthMask</strong> commands. </p>

<p>I seem to vaguely remember from several years ag that a multi-pass rendering may be required. If so, could someone point me to how to accomplish this with the proper set of command parameters?</p>

<p>thanks in advance.</p>

<hr>

<p>This is an example of what I'm expecting - The cylinder should cut a hole through the frustum. I've rendered it here by trimming the black cylinder within the frustum so that it just just out past the frustum surface.</p>

<p><a href="https://i.sstatic.net/vcxON.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/vcxON.jpg" alt="enter image description here"></a></p>

CC BY-SA 3.0
2017-09-13T18:08:29.897Z
I´d treat the cylinder as a "light source", putting its geometry as a shader uniform and test fragment position against the cylinder, clipping fragments inside. Not sure if it is feasible for you or even interesting but in case you or others are stuck I´m just saying there is a way :-)
Andreas
CC BY-SA 3.0
2017-09-13T18:10:15.700Z
FYI: you have not enabled depth test
Andreas
CC BY-SA 3.0
2017-06-29T16:28:03.307Z
Dan, I've added an image describing my expectation as an "answer".
Sharat V Chandrasekhar
CC BY-SA 3.0
2017-06-29T16:43:57.417Z
You probably want to search for CSG (Constructive Solid Geometry) and Stencils.  For example, this https://www.usenix.org/legacy/events/usenix05/tech/freenix/full_papers/kirsch/kirsch.pdf might allow you to achieve the desired results.  Similarly, perhaps look at http://www.opencsg.org/ ?

Actually, I might add this as a possible answer if no other answers appear
Simon F
CC BY-SA 3.0
2017-06-29T16:50:45.123Z
Thanks! In fact, I was reading that paper last night, but was hoping that OpenGL would have provided a way of accomplishing this with just basic stencil functions. It looks like quite an involved task!
Sharat V Chandrasekhar
CC BY-SA 3.0
2017-06-29T17:49:04.300Z
Simon, one of the issues that I'd have with using the EXCELLENT CSG resources is that I'm accessing OpenGL via Visual Basic modules linked to an Excel spreadsheet from which I drive my application. I do not know whether or not the project files for MSVC6 will include extensions for VBA.
Sharat V Chandrasekhar
<opengl><stencil-test>
Honouring Depth Test with Stencil Test
Sharat V Chandrasekhar
# Honouring Depth Test with Stencil Test

(Asked by: username_0 on 2017-06-28)

I'm using OpenGL's stencil test to carve out a semicircular hole in a transition frustum with a cylinder oriented perpendicularly. However as one can see below, while this objective is achieved, the cylinder is still visible through the frustum which shouldn't be the case.

[![enter image description here](https://i.sstatic.net/pmWBD.jpg)](https://i.sstatic.net/pmWBD.jpg)

I'm using the following code fragment that I found at
<https://en.wikibooks.org/wiki/OpenGL_Programming/Mini-Portal>

```
glClear(GL_DEPTH_BUFFER_BIT);
glEnable(GL_STENCIL_TEST);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glDepthMask(GL_FALSE);
glStencilFunc(GL_NEVER, 1, 0xFF);
glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP);  // draw 1s on test fail (always)

// draw stencil pattern
glStencilMask(0xFF);
glClear(GL_STENCIL_BUFFER_BIT);  // needs mask=0xFF
DRAW_CYLINDER ();

glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glDepthMask(GL_TRUE);
glStencilMask(0x00);
// draw where stencil's value is 0
glStencilFunc(GL_EQUAL, 0, 0xFF);
/* (nothing to draw) */
// draw only where stencil's value is 1
glStencilFunc(GL_EQUAL, 1, 0xFF);

DRAW_FRUSTUM();

glDisable(GL_STENCIL_TEST);

```

Moreover I've found that the results are insensitive to the parameters of the **glStencilOp** and **DepthMask** commands.

I seem to vaguely remember from several years ag that a multi-pass rendering may be required. If so, could someone point me to how to accomplish this with the proper set of command parameters?

thanks in advance.

---

This is an example of what I'm expecting - The cylinder should cut a hole through the frustum. I've rendered it here by trimming the black cylinder within the frustum so that it just just out past the frustum surface.

[![enter image description here](https://i.sstatic.net/vcxON.jpg)](https://i.sstatic.net/vcxON.jpg)

**Tags:** opengl, stencil-test

> **Comments:**
> **username_0**: Dan, I've added an image describing my expectation as an "answer".  
> **username_2**: You probably want to search for CSG (Constructive Solid Geometry) and Stencils.  For example, this https://www.usenix.org/legacy/events/usenix05/tech/freenix/full_papers/kirsch/kirsch.pdf might allow you to achieve the desired results.  Similarly, perhaps look at http://www.opencsg.org/ ?  Actually, I might add this as a possible answer if no other answers appear  
> **username_0**: Thanks! In fact, I was reading that paper last night, but was hoping that OpenGL would have provided a way of accomplishing this with just basic stencil functions. It looks like quite an involved task!  
> **username_0**: username_2, one of the issues that I'd have with using the EXCELLENT CSG resources is that I'm accessing OpenGL via Visual Basic modules linked to an Excel spreadsheet from which I drive my application. I do not know whether or not the project files for MSVC6 will include extensions for VBA.  
> **username_1**: I´d treat the cylinder as a "light source", putting its geometry as a shader uniform and test fragment position against the cylinder, clipping fragments inside. Not sure if it is feasible for you or even interesting but in case you or others are stuck I´m just saying there is a way :-)  
> **username_1**: FYI: you have not enabled depth test  

---

## 1 answers

### Answer by username_1 on 2017-09-13. Score: 1

When cylinder and frustum intersect the hole will be a view into the internals of the cylinder. **The hole is where external of the cylinder occlude frustum and internal of the cylinder does not occlude the frustum.**

Knowing this you should simply do the following to find the surface (or rather volume) made from the hole.

Render frustum without color and set depth accordingly. Set a stencil bit where the cylinder (external) occludes the frustum, without setting new depth. Clear the previously set bit where the internal of the cylinder occludes the frustum, without setting new depth.

Alternate between internal and external using CullFace.

The stencil mask you now have is the surface where the frustum hole is. Render the frustum (in color) masking it with the stencil. Make sure to use LEQUALS depth test to allow the frustum to render on top of itself. Voila. Note that the depth buffer holds the value of the unmasked frustum so If you need to render something within the hole you would have to render the whole scene again and mask it to only render in the hole unless depth test passes against the frustum. Left as student exercise.

> **Comments:**
> **username_0**: Sorry for this late acknowledgement, but I had to shelve this project owing to other more pressing priorities, but now that I've some time to return to it, I'll investigate your suggestion. Thanks a ton!  

---

