<p>With your question you have basically identified a core problem in your assumption of <em>&quot;how the OpenGL's gluUnProject works&quot;</em>, namely that this function <em>isn't</em> actually part of the OpenGL. The GLU library is a (very old) auxiliary library containing some useful methods (of questionable relevance nowadays) for computer graphics computing on top of OpenGL.</p>
<p>Now why does this matter? Well, those <strong>GLU functions have absolutely nothing to do with hardware accelerated rendering</strong>, which is the core, if not <em>the only</em>, competence of OpenGL. So they neither have any business messing with the hardware resources you'd need for your implementation of the method, namely reading back the hardware depth buffer. In fact <code>gluUnProject</code> is a rather simple method containing a few simple 4x4 matrix multiplies, that's really all.</p>
<p>You certainly <em>can</em> use your method for determining the actual 3D point of a pixel, but you'd have to do it yourself on top of that computation. This way you'd also be <strong>responsible yourself for reading back the depth buffer</strong> onto the CPU, which <strong>isn't actually a zero-cost operation</strong>, even for a single pixel, since it causes pipeline stalls and many other synchronization issues. That's why it might not be the best idea to have such a basic operation do it on its own, when often a general ray without actual depth information is enough.</p>
<p>So while <code>gluUnProject</code>, at least back in the day when rendering was all simple like that, could theoretically do a <code>glReadPixels</code> call internally, that's just not in its area of competence as a simple geometric linear algebra function. And nowadays you'd not use GLU anymore anyway, rather than the third-party mathematics library you'd have to use on top of OpenGL (since modern OpenGL doesn't do matrix math for you anymore). And such third-party libaries, while possibly offering a similar function as <code>gluUnProject</code>, would be even more out of their depths (pun intended) to call OpenGL functions willy-nilly and query hardware information that's absolutely not of their responsibility to mess with. Not to speak of the fact that rendering nowadays is a lot more flexible and your depth buffer data could be computed in any or no way possible.</p>

CC BY-SA 4.0
2022-06-10T11:37:47.670Z
2022-06-10T11:37:47.670Z
Christian Rau
<p>I was studying how the OpenGL's <a href="https://dondi.lmu.build/share/cg/unproject-explained.pdf" rel="nofollow noreferrer">gluUnProject</a> works. There is one thing I don't understand: is there any benefit of returning a ray instead of 3D point?</p>
<p>One can use this function to get a ray not a 3D point by inputting screen x and y. The linked document said that <code>we can’t really go from a mouse point to a single  3D point</code>.</p>
<p>But I think it is still possible to get a 3D point from mouse coordinates. 3D graphics libraries usually implement z buffer and we can easily get the value of z (in screen coordinates) from z buffer. For example, given a mouse coordinates x and y, the value of z can be calculated by <code>1.0/zbuffer[y][x]</code>. It is easy to get 3D world space x, y and z with all these information. So why <code>gluUnProject</code> is not implemented this way?</p>

<opengl><z-buffer>
Understanding gluUnProject
Jimmy Yang
# Understanding gluUnProject

(Asked by: username_0 on 2022-06-10)

I was studying how the OpenGL's [gluUnProject](https://dondi.lmu.build/share/cg/unproject-explained.pdf) works. There is one thing I don't understand: is there any benefit of returning a ray instead of 3D point?

One can use this function to get a ray not a 3D point by inputting screen x and y. The linked document said that `we can’t really go from a mouse point to a single 3D point`.

But I think it is still possible to get a 3D point from mouse coordinates. 3D graphics libraries usually implement z buffer and we can easily get the value of z (in screen coordinates) from z buffer. For example, given a mouse coordinates x and y, the value of z can be calculated by `1.0/zbuffer[y][x]`. It is easy to get 3D world space x, y and z with all these information. So why `gluUnProject` is not implemented this way?

**Tags:** opengl, z-buffer


---

## 1 answers

### Answer by username_1 on 2022-06-10. Score: 1 | Accepted answer

With your question you have basically identified a core problem in your assumption of *"how the OpenGL's gluUnProject works"*, namely that this function *isn't* actually part of the OpenGL. The GLU library is a (very old) auxiliary library containing some useful methods (of questionable relevance nowadays) for computer graphics computing on top of OpenGL.

Now why does this matter? Well, those **GLU functions have absolutely nothing to do with hardware accelerated rendering**, which is the core, if not *the only*, competence of OpenGL. So they neither have any business messing with the hardware resources you'd need for your implementation of the method, namely reading back the hardware depth buffer. In fact `gluUnProject` is a rather simple method containing a few simple 4x4 matrix multiplies, that's really all.

You certainly *can* use your method for determining the actual 3D point of a pixel, but you'd have to do it yourself on top of that computation. This way you'd also be **responsible yourself for reading back the depth buffer** onto the CPU, which **isn't actually a zero-cost operation**, even for a single pixel, since it causes pipeline stalls and many other synchronization issues. That's why it might not be the best idea to have such a basic operation do it on its own, when often a general ray without actual depth information is enough.

So while `gluUnProject`, at least back in the day when rendering was all simple like that, could theoretically do a `glReadPixels` call internally, that's just not in its area of competence as a simple geometric linear algebra function. And nowadays you'd not use GLU anymore anyway, rather than the third-party mathematics library you'd have to use on top of OpenGL (since modern OpenGL doesn't do matrix math for you anymore). And such third-party libaries, while possibly offering a similar function as `gluUnProject`, would be even more out of their depths (pun intended) to call OpenGL functions willy-nilly and query hardware information that's absolutely not of their responsibility to mess with. Not to speak of the fact that rendering nowadays is a lot more flexible and your depth buffer data could be computed in any or no way possible.


---

