<p>Better late than never.
I made a small demo project with ray marching on AVP: <a href="https://github.com/murinson/MetalRaymarch" rel="nofollow noreferrer">https://github.com/murinson/MetalRaymarch</a>
In order for my ray marching shader to be executed for every pixel, I place a large sphere at the origin with front face culling. This sphere’s uv coordinates are used to calculate the ray direction.
If you found a different approach, I’d be very interested to learn about it!</p>

CC BY-SA 4.0
2024-12-26T22:41:42.730Z
2024-12-26T22:41:42.730Z
user22797
<p>I am trying to calculate the ray direction for each pixel in a post processing shader on the Vision Pro for raymarching. I am using the inverse of the projection matrix created from the tangents provided by the Vision OS API to multiply the screen's coordinate converted to NDC. This works perfectly on the simulator and matches the raster scene rendered with that projection matrix, however on the device, the projected rays are completely off from the raster scene (they seem warped at the edges and don't match the projection).</p>
<p>This is also the case if I calculate the rays directly from the FOV &amp; aspect without using the projection matrix inverse.</p>
<p>From what I understand the projection frustum tangents are skewed somewhat in the VR projection, but i thought the inverse of the projection matrix should still unproject the coordinate correctly.</p>
<p>Does anyone have any experience with raycasting/raymarching in VR and could shed some light?</p>
<p>Thank you.</p>

<projections><raymarching><virtual-reality>
Ray direction in VR (Vision Pro)
toastie
# Ray direction in VR (Vision Pro)

(Asked by: username_0 on 2024-07-31)

I am trying to calculate the ray direction for each pixel in a post processing shader on the Vision Pro for raymarching. I am using the inverse of the projection matrix created from the tangents provided by the Vision OS API to multiply the screen's coordinate converted to NDC. This works perfectly on the simulator and matches the raster scene rendered with that projection matrix, however on the device, the projected rays are completely off from the raster scene (they seem warped at the edges and don't match the projection).

This is also the case if I calculate the rays directly from the FOV & aspect without using the projection matrix inverse.

From what I understand the projection frustum tangents are skewed somewhat in the VR projection, but i thought the inverse of the projection matrix should still unproject the coordinate correctly.

Does anyone have any experience with raycasting/raymarching in VR and could shed some light?

Thank you.

**Tags:** projections, raymarching, virtual-reality


---

## 1 answers

### Answer by username_1 on 2024-12-26. Score: 0

Better late than never.
I made a small demo project with ray marching on AVP: <https://github.com/murinson/MetalRaymarch>
In order for my ray marching shader to be executed for every pixel, I place a large sphere at the origin with front face culling. This sphere’s uv coordinates are used to calculate the ray direction.
If you found a different approach, I’d be very interested to learn about it!


---

