  Light plays an important role in human well-being. However, most computer
vision tasks treat pixels without considering their relationship to physical
luminance. To address this shortcoming, we introduce the Laval Photometric
Indoor HDR Dataset, the first large-scale photometrically calibrated dataset of
high dynamic range 360{\deg} panoramas. Our key contribution is the calibration
of an existing, uncalibrated HDR Dataset. We do so by accurately capturing RAW
bracketed exposures simultaneously with a professional photometric measurement
device (chroma meter) for multiple scenes across a variety of lighting
conditions. Using the resulting measurements, we establish the calibration
coefficients to be applied to the HDR images. The resulting dataset is a rich
representation of indoor scenes which displays a wide range of illuminance and
color, and varied types of light sources. We exploit the dataset to introduce
three novel tasks, where: per-pixel luminance, per-pixel color and planar
illuminance can be predicted from a single input image. Finally, we also
capture another smaller photometric dataset with a commercial 360{\deg} camera,
to experiment on generalization across cameras. We are optimistic that the
release of our datasets and associated code will spark interest in physically
accurate light estimation within the community. Dataset and code are available
at https://lvsn.github.io/beyondthepixel/.
