<p>You're basically mixing up your spaces.</p>

<p>If you look at the <a href="http://doc.qt.io/qt-5/qvector3d.html#unproject" rel="nofollow noreferrer">documentation for <code>QVector3D::unproject</code></a>, you see that it actually <strong>expects coordinates in <em>window space</em></strong>. And this makes sense, since you explicitly query the viewport (i.e. basically the window's <em>pixel</em> dimensions) and give that to the function.</p>

<p>But with your ratio computation above you already transformed your coordinates from that space into normalized device space, i.e. the $[-1,1]$ square. So this whole computation isn't necessary at all and is already done for you by <code>unproject</code>. Likewise are the near and far distances from world/eye space not the same in window space, which uses the normalized $[0,1]$ depth range. If we assume that your viewport is your whole window, then all you need to do is put those pixel coordinates you clicked on into <code>unproject</code> (still accounting for the different Y-convention, though):</p>

<pre><code>dir = QVector3D(event.x, height()-event.y, 0.);    //in window space 0 = near, 1 = far
dir = dir.unproject(modelViewMatrix, projectionMatrix, vp);
</code></pre>

<p>A problem is that you <strong>don't easily know the window space coordinates of the camera position</strong>, since it is in front of the near plane, but the transformation from $[0,1]$ to $[near,far]$ is non-linear. One way to overcome this would be to just unproject a point on the near plane (depth = 0) as origin and the difference to the unproject of that same point on the far plane (depth = 1) as the direction.</p>

<pre><code>QVector3D near(QVector3D(event.x, height()-event.y, 0.);
QVector3D far( QVector3D(event.x, height()-event.y, 1.);
origin = near.unproject(modelViewMatrix, projectionMatrix, vp);
dir = far.unproject(modelViewMatrix, projectionMatrix, vp) - origin;
</code></pre>

<p>Or you compute <code>dir</code> just the same and don't unproject the origin at all, rather than just transforming the eye-space origin (which is conveniently $\mathbf{0}$) back into model space (assuming your modelview matrix isn't projective, which it really shouldn't be, otherwise don't forget the W-division):</p>

<pre><code>origin = QVector3D(modelViewMatrix.inverted() * QVector4D(0., 0., 0., 1.));
//origin = QVector3D(modelViewMatrix.inverted().column(3));    //would work the same
</code></pre>

<p>If your viewport is <em>not</em> the same as your window dimensions however, you need to fiddle with the mouse coordinates a little more and transform them from the entire window into your viewport rect before unprojecting them. But the point is that </p>

<h1><code>unproject</code> expects window space (i.e. pixel) coordinates.</h1>

CC BY-SA 3.0
2017-10-01T17:24:39.123Z
2017-10-02T10:24:15.617Z
2017-10-02T10:24:15.617Z
Christian Rau
<p>I am struggling trying to implement vertex selection by mouse clicking. I load in some set of vertices and display them onto the screen. Then I do the following to try and retrieve a line that passes through the selected point in world coordinates and through the eye position.</p>

<pre><code>  setFocus();
  float xRatio, yRatio, xScene, yScene;
  xRatio = (float)event-&gt;x() / width();
  yRatio = (float)event-&gt;y() / height();

  xScene = (1-xRatio)*-1 + xRatio*1;
  yScene = yRatio*-1 + (1-yRatio)*1;
  // mouse click now transformed to [-1,1]^2

  // Import the viewPort
      GLint *params = new GLint(4);
      glGetIntegerv(GL_VIEWPORT, params);
      QRect vp = QRect(*params, *(params + 1), *(params + 2), *(params + 3));     


      QVector3D dir = QVector3D(xScene, yScene, 0.2); // distance to near pane = 0.2
      QVector3D origin = QVector3D(0.0, 0.0, 0.0);

  // Now I have the origin and point in near clipping pane of the line (I think)
  // Unproject them to world coordinates.
      dir = dir.unproject(modelViewMatrix, projectionMatrix, vp);
      origin = origin.unproject(modelViewMatrix, projectionMatrix, vp);

  float minDist = 1000.0;
  int minIndex = 0;
  float dist;

  for (int i = 0; i &lt; vertexCoords.size(); ++i){
      dist = vertexCoords[i].distanceToLine(origin, dir);
      if (dist &lt; minDist){
          minDist = dist;
          minIndex = i;
      }
  }
</code></pre>

<p>I keep getting wrong result with this method. It will almost always select the same point and report a large distance of about 20 to that point. So something is going terribly wrong. Can anyone spot what is going wrong or what I am missing?</p>

<p>Additionally, here is how the view and projection matrices look:</p>

<pre><code>  modelViewMatrix.setToIdentity();
  modelViewMatrix.translate(QVector3D(0.0, 0.0, -3.0));
  modelViewMatrix.scale(QVector3D(1.0, 1.0, 1.0));
  modelViewMatrix.rotate(rotX, 0.0, 1.0, 0.0);
  modelViewMatrix.rotate(rotY, 1.0, 0.0, 0.0);

  projectionMatrix.setToIdentity();
  projectionMatrix.perspective(FoV, dispRatio, 0.2, 8.0);
</code></pre>

<p>Sorry for the long question, I have found quite some resources online but none of them help me figure out what is going wrong. Also, if there is a better method I would gladly hear it! Thanks</p>

<opengl>
How to perform vertex selection using unproject (Qt/openGL)
Slugger
# How to perform vertex selection using unproject (Qt/openGL)

(Asked by: username_0 on 2016-12-06)

I am struggling trying to implement vertex selection by mouse clicking. I load in some set of vertices and display them onto the screen. Then I do the following to try and retrieve a line that passes through the selected point in world coordinates and through the eye position.

```
  setFocus();
  float xRatio, yRatio, xScene, yScene;
  xRatio = (float)event->x() / width();
  yRatio = (float)event->y() / height();

  xScene = (1-xRatio)*-1 + xRatio*1;
  yScene = yRatio*-1 + (1-yRatio)*1;
  // mouse click now transformed to [-1,1]^2

  // Import the viewPort
      GLint *params = new GLint(4);
      glGetIntegerv(GL_VIEWPORT, params);
      QRect vp = QRect(*params, *(params + 1), *(params + 2), *(params + 3));     

      QVector3D dir = QVector3D(xScene, yScene, 0.2); // distance to near pane = 0.2
      QVector3D origin = QVector3D(0.0, 0.0, 0.0);

  // Now I have the origin and point in near clipping pane of the line (I think)
  // Unproject them to world coordinates.
      dir = dir.unproject(modelViewMatrix, projectionMatrix, vp);
      origin = origin.unproject(modelViewMatrix, projectionMatrix, vp);

  float minDist = 1000.0;
  int minIndex = 0;
  float dist;

  for (int i = 0; i < vertexCoords.size(); ++i){
      dist = vertexCoords[i].distanceToLine(origin, dir);
      if (dist < minDist){
          minDist = dist;
          minIndex = i;
      }
  }

```

I keep getting wrong result with this method. It will almost always select the same point and report a large distance of about 20 to that point. So something is going terribly wrong. Can anyone spot what is going wrong or what I am missing?

Additionally, here is how the view and projection matrices look:

```
  modelViewMatrix.setToIdentity();
  modelViewMatrix.translate(QVector3D(0.0, 0.0, -3.0));
  modelViewMatrix.scale(QVector3D(1.0, 1.0, 1.0));
  modelViewMatrix.rotate(rotX, 0.0, 1.0, 0.0);
  modelViewMatrix.rotate(rotY, 1.0, 0.0, 0.0);

  projectionMatrix.setToIdentity();
  projectionMatrix.perspective(FoV, dispRatio, 0.2, 8.0);

```

Sorry for the long question, I have found quite some resources online but none of them help me figure out what is going wrong. Also, if there is a better method I would gladly hear it! Thanks

**Tags:** opengl


---

## 1 answers

### Answer by username_1 on 2017-10-01. Score: 2

You're basically mixing up your spaces.

If you look at the [documentation for `QVector3D::unproject`](http://doc.qt.io/qt-5/qvector3d.html#unproject), you see that it actually **expects coordinates in *window space***. And this makes sense, since you explicitly query the viewport (i.e. basically the window's *pixel* dimensions) and give that to the function.

But with your ratio computation above you already transformed your coordinates from that space into normalized device space, i.e. the $[-1,1]$ square. So this whole computation isn't necessary at all and is already done for you by `unproject`. Likewise are the near and far distances from world/eye space not the same in window space, which uses the normalized $[0,1]$ depth range. If we assume that your viewport is your whole window, then all you need to do is put those pixel coordinates you clicked on into `unproject` (still accounting for the different Y-convention, though):

```
dir = QVector3D(event.x, height()-event.y, 0.);    //in window space 0 = near, 1 = far
dir = dir.unproject(modelViewMatrix, projectionMatrix, vp);

```

A problem is that you **don't easily know the window space coordinates of the camera position**, since it is in front of the near plane, but the transformation from $[0,1]$ to $[near,far]$ is non-linear. One way to overcome this would be to just unproject a point on the near plane (depth = 0) as origin and the difference to the unproject of that same point on the far plane (depth = 1) as the direction.

```
QVector3D near(QVector3D(event.x, height()-event.y, 0.);
QVector3D far( QVector3D(event.x, height()-event.y, 1.);
origin = near.unproject(modelViewMatrix, projectionMatrix, vp);
dir = far.unproject(modelViewMatrix, projectionMatrix, vp) - origin;

```

Or you compute `dir` just the same and don't unproject the origin at all, rather than just transforming the eye-space origin (which is conveniently $\mathbf{0}$) back into model space (assuming your modelview matrix isn't projective, which it really shouldn't be, otherwise don't forget the W-division):

```
origin = QVector3D(modelViewMatrix.inverted() * QVector4D(0., 0., 0., 1.));
//origin = QVector3D(modelViewMatrix.inverted().column(3));    //would work the same

```

If your viewport is *not* the same as your window dimensions however, you need to fiddle with the mouse coordinates a little more and transform them from the entire window into your viewport rect before unprojecting them. But the point is that

# `unproject` expects window space (i.e. pixel) coordinates.


---

