  Lacking realistic ground truth data, image denoising techniques are
traditionally evaluated on images corrupted by synthesized i.i.d. Gaussian
noise. We aim to obviate this unrealistic setting by developing a methodology
for benchmarking denoising techniques on real photographs. We capture pairs of
images with different ISO values and appropriately adjusted exposure times,
where the nearly noise-free low-ISO image serves as reference. To derive the
ground truth, careful post-processing is needed. We correct spatial
misalignment, cope with inaccuracies in the exposure parameters through a
linear intensity transform based on a novel heteroscedastic Tobit regression
model, and remove residual low-frequency bias that stems, e.g., from minor
illumination changes. We then capture a novel benchmark dataset, the Darmstadt
Noise Dataset (DND), with consumer cameras of differing sensor sizes. One
interesting finding is that various recent techniques that perform well on
synthetic noise are clearly outperformed by BM3D on photographs with real
noise. Our benchmark delineates realistic evaluation scenarios that deviate
strongly from those commonly used in the scientific literature.
