<p>Nicol Bolas explains why you can't expect pixel-accurate reproductions, but that doesn't mean you can't test with screenshots.</p>

<p>Don't compare the screenshots byte-for-byte. Instead, use a <em>perceptual diff</em> to quantify how different the renders are. You can then set a threshold in your test and have the test fail if the renders are too different. A harsh threshold will let you filter out the rounding differences between platforms, and the slight differences around object silhouettes, but still detect objects in the wrong place or inconsistent texturing. A more forgiving threshold will allow bigger differences in texturing (or even with different shader versions or extensions, if you have different rendering paths for different hardware), and only complain if you have gross differences such as missing lights or a completely different scene being rendered.</p>

<p>The <a href="http://pdiff.sourceforge.net/" rel="noreferrer">pdiff tool</a> is commonly used for this task, and I've set up test systems based on it before.</p>

CC BY-SA 3.0
2018-03-28T15:38:56.060Z
2018-03-28T15:38:56.060Z
Dan Hulme
<p>What you want cannot reasonably be done. No graphics API defines byte-accurate results. Every implementation will have its own variances, which will be visible in the results. OpenGL provides no insurance that you will get invariant results across implementations.</p>

<p>At the end of the day, you can't do testing in this fashion.</p>

CC BY-SA 3.0
2018-03-28T15:16:58.927Z
2018-03-28T15:16:58.927Z
Nicol Bolas
<p>I have some OpenGL based software and I would like to set up some automated testing using image comparison, but one problem we always have is that the screenshots we take always have some small differences across machines, usually due to graphics driver difference or something else (?).  </p>

<p>What strategies are there for getting consistent screenshots across machines with varying hardware and graphics drivers?</p>

CC BY-SA 3.0
2018-03-28T13:50:11.653Z
What sort of differences?
gallickgunner
CC BY-SA 3.0
2018-03-28T16:08:40.193Z
Usually the differences were like little outlines around objects, like if anti aliasing were on one but off on the other. But we turned anti aliasing off always when running the tests. And it didn't always happen, just very rarely with certain objects.
default
<opengl><opengl-es>
How to take consistent screenshots across multiple machines?
default
# How to take consistent screenshots across multiple machines?

(Asked by: username_0 on 2018-03-28)

I have some OpenGL based software and I would like to set up some automated testing using image comparison, but one problem we always have is that the screenshots we take always have some small differences across machines, usually due to graphics driver difference or something else (?).

What strategies are there for getting consistent screenshots across machines with varying hardware and graphics drivers?

**Tags:** opengl, opengl-es

> **Comments:**
> **username_1**: What sort of differences?  
> **username_0**: Usually the differences were like little outlines around objects, like if anti aliasing were on one but off on the other. But we turned anti aliasing off always when running the tests. And it didn't always happen, just very rarely with certain objects.  

---

## 2 answers

### Answer by username_2 on 2018-03-28. Score: 5

username_3 username_3 explains why you can't expect pixel-accurate reproductions, but that doesn't mean you can't test with screenshots.

Don't compare the screenshots byte-for-byte. Instead, use a *perceptual diff* to quantify how different the renders are. You can then set a threshold in your test and have the test fail if the renders are too different. A harsh threshold will let you filter out the rounding differences between platforms, and the slight differences around object silhouettes, but still detect objects in the wrong place or inconsistent texturing. A more forgiving threshold will allow bigger differences in texturing (or even with different shader versions or extensions, if you have different rendering paths for different hardware), and only complain if you have gross differences such as missing lights or a completely different scene being rendered.

The [pdiff tool](http://pdiff.sourceforge.net/) is commonly used for this task, and I've set up test systems based on it before.


---

### Answer by username_3 on 2018-03-28. Score: 2

What you want cannot reasonably be done. No graphics API defines byte-accurate results. Every implementation will have its own variances, which will be visible in the results. OpenGL provides no insurance that you will get invariant results across implementations.

At the end of the day, you can't do testing in this fashion.


---

