<p>Ideally, motion-to-photon latency is exactly what it says. It measures time from when the user physically makes a <em>motion</em> (say, begins to turn their head) to when the <em>photons</em> physically coming out of the display change to reflect that motion.</p>

<p>With a bit of apparatus, this can be measured directly—for instance by using a high-speed camera aimed at the headset. You give the headset a light bump with your finger (or use some automated device to bump it), and you can see in the high-speed video when the motion occurred and when the image visible on the screen updated to reflect the motion.</p>

<p>The complication leading to different definitions occurs because many VR headsets have <em>rolling displays</em>, where the physical pixels on the display light up progressively, one row at a time, as the image is scanned out from the host PC. So, when you define motion-to-photon latency, you have to say <em>which</em> photons you're talking about. The very first ones to scan out? The very last ones? Somewhere in the middle?</p>

<p>You can pretty much make whichever definition you want here about what point in the scanout process to use as your reference. There may be an argument for one definition or another based on psychovisual effects (i.e. how much of your visual field must be updated in order for your brain to perceive motion?) but this is beyond my knowledge.</p>

<p>On the other hand, if a headset has a global display instead of a rolling one, then the pixels are kept dark while being updated and then the whole display is flashed on and off at the end of scanout. In this case it's clear that the appropriate reference point to measure is the beginning of the flash, since all the photons come out at the same time.</p>

<p>Finally, it's worth noting that latency measurements incorporated into an app or API are approximations. The software can't observe the motion nor the photons directly, so instead we must use software-visible proxy events.</p>

<ul>
<li>The app querying for the latest head pose is a proxy for user motion, but note that this discards any latency in the sensors, USB driver, sensor fusion processing, and time the data spent sitting around in a buffer waiting for the app to query it.</li>
<li>The GPU flipping its display buffers is a proxy for photons beginning to come out, but this discards any latency added in the scanout and display process.</li>
</ul>

<p>So, ideally an app-measured latency figure should include some corrections to at least estimate the above factors. Or accept that you're not really measuring the full motion-to-photon latency, but just the app-query-to-flip latency or some such.</p>

CC BY-SA 3.0
2016-08-20T23:23:03.777Z
2016-08-20T23:23:03.777Z
Nathan Reed
<p>The rough definition of <em>"MTP"</em> is clear. But the exact one is not. You can refer to these links:</p>

<ul>
<li><a href="http://www.chioka.in/what-is-motion-to-photon-latency" rel="nofollow noreferrer">What is Motion-To-Photon Latency? | chioka.in</a></li>
<li><a href="http://xinreality.com/wiki/Motion-to-photon_latency" rel="nofollow noreferrer">Motion-to-photon latency | xinreality.com/wiki</a>.</li>
</ul>

<p>But from different online articles, we get different definitions regarding how the value is measured:</p>

<ol>
<li><p>From <a href="https://developer.oculus.com/blog/optimizing-vr-graphics-with-late-latching" rel="nofollow noreferrer">Atman Binstock (Oculus) blog</a>, we can see:<br/>
<code>MTP = T2(time to flip) - T1(Time of PO)</code>.<br/>
But, <code>T2</code> is just beginning to display the image, not finish displaying.</p></li>
<li><p>From <a href="http://www.chioka.in/what-is-motion-to-photon-latency/" rel="nofollow noreferrer">John Carmack (Oculus) blog</a>:<br/>
<code>MTP = T2(finish displaying) - T1(Time of PO)</code>.</p></li>
<li><p>From Alex Vlachos (Valve) slides: <em>Advanced VR Rendering</em>, GDC 2015:<br/>
<code>MTP = T2(finish displaying) - T1(Time of PO)</code></p></li>
<li><p>From <a href="http://www.qualcomm.cn/news/blog-2016-07-20" rel="nofollow noreferrer">Qualcomm blog</a> (sorry it is in Chinese):<br/>
<code>MTP = T2(half of finish displaying) - T1(Time of PO)</code></p></li>
</ol>

<p>I think Qualcomm's explanation is the best. If it is correct, we can relax the rendering time budget. What is your opinion?</p>

CC BY-SA 3.0
2016-08-22T00:34:50.647Z
@Julien Guertault, Thank you for refining the format. I don't know how to do that:)
Hao Zhang
CC BY-SA 3.0
2016-08-22T02:16:58.200Z
At your service. :) Here's an overview if you're interested: http://stackoverflow.com/editing-help
Julien Guertault
<rendering><virtual-reality>
What is the definition of "motion to photon" in VR?
Hao Zhang
# What is the definition of "motion to photon" in VR?

(Asked by: username_0 on 2016-08-08)

The rough definition of *"MTP"* is clear. But the exact one is not. You can refer to these links:

* [What is Motion-To-Photon Latency? | chioka.in](http://www.chioka.in/what-is-motion-to-photon-latency)
* [Motion-to-photon latency | xinreality.com/wiki](http://xinreality.com/wiki/Motion-to-photon_latency).

But from different online articles, we get different definitions regarding how the value is measured:

1. From [Atman Binstock (Oculus) blog](https://developer.oculus.com/blog/optimizing-vr-graphics-with-late-latching), we can see:  
   `MTP = T2(time to flip) - T1(Time of PO)`.  
   But, `T2` is just beginning to display the image, not finish displaying.
2. From [John Carmack (Oculus) blog](http://www.chioka.in/what-is-motion-to-photon-latency/):  
   `MTP = T2(finish displaying) - T1(Time of PO)`.
3. From Alex Vlachos (Valve) slides: *Advanced VR Rendering*, GDC 2015:  
   `MTP = T2(finish displaying) - T1(Time of PO)`
4. From [Qualcomm blog](http://www.qualcomm.cn/news/blog-2016-07-20) (sorry it is in Chinese):  
   `MTP = T2(half of finish displaying) - T1(Time of PO)`

I think Qualcomm's explanation is the best. If it is correct, we can relax the rendering time budget. What is your opinion?

**Tags:** rendering, virtual-reality

> **Comments:**
> **username_0**: @username_1, Thank you for refining the format. I don't know how to do that:)  
> **username_1**: At your service. :) Here's an overview if you're interested: http://stackoverflow.com/editing-help  

---

## 1 answers

### Answer by username_2 on 2016-08-20. Score: 6 | Accepted answer

Ideally, motion-to-photon latency is exactly what it says. It measures time from when the user physically makes a *motion* (say, begins to turn their head) to when the *photons* physically coming out of the display change to reflect that motion.

With a bit of apparatus, this can be measured directly—for instance by using a high-speed camera aimed at the headset. You give the headset a light bump with your finger (or use some automated device to bump it), and you can see in the high-speed video when the motion occurred and when the image visible on the screen updated to reflect the motion.

The complication leading to different definitions occurs because many VR headsets have *rolling displays*, where the physical pixels on the display light up progressively, one row at a time, as the image is scanned out from the host PC. So, when you define motion-to-photon latency, you have to say *which* photons you're talking about. The very first ones to scan out? The very last ones? Somewhere in the middle?

You can pretty much make whichever definition you want here about what point in the scanout process to use as your reference. There may be an argument for one definition or another based on psychovisual effects (i.e. how much of your visual field must be updated in order for your brain to perceive motion?) but this is beyond my knowledge.

On the other hand, if a headset has a global display instead of a rolling one, then the pixels are kept dark while being updated and then the whole display is flashed on and off at the end of scanout. In this case it's clear that the appropriate reference point to measure is the beginning of the flash, since all the photons come out at the same time.

Finally, it's worth noting that latency measurements incorporated into an app or API are approximations. The software can't observe the motion nor the photons directly, so instead we must use software-visible proxy events.

* The app querying for the latest head pose is a proxy for user motion, but note that this discards any latency in the sensors, USB driver, sensor fusion processing, and time the data spent sitting around in a buffer waiting for the app to query it.
* The GPU flipping its display buffers is a proxy for photons beginning to come out, but this discards any latency added in the scanout and display process.

So, ideally an app-measured latency figure should include some corrections to at least estimate the above factors. Or accept that you're not really measuring the full motion-to-photon latency, but just the app-query-to-flip latency or some such.


---

