  Color constancy is the recovery of true surface color from observed color,
and requires estimating the chromaticity of scene illumination to correct for
the bias it induces. In this paper, we show that the per-pixel color statistics
of natural scenes---without any spatial or semantic context---can by themselves
be a powerful cue for color constancy. Specifically, we describe an illuminant
estimation method that is built around a "classifier" for identifying the true
chromaticity of a pixel given its luminance (absolute brightness across color
channels). During inference, each pixel's observed color restricts its true
chromaticity to those values that can be explained by one of a candidate set of
illuminants, and applying the classifier over these values yields a
distribution over the corresponding illuminants. A global estimate for the
scene illuminant is computed through a simple aggregation of these
distributions across all pixels. We begin by simply defining the
luminance-to-chromaticity classifier by computing empirical histograms over
discretized chromaticity and luminance values from a training set of natural
images. These histograms reflect a preference for hues corresponding to smooth
reflectance functions, and for achromatic colors in brighter pixels. Despite
its simplicity, the resulting estimation algorithm outperforms current
state-of-the-art color constancy methods. Next, we propose a method to learn
the luminance-to-chromaticity classifier "end-to-end". Using stochastic
gradient descent, we set chromaticity-luminance likelihoods to minimize errors
in the final scene illuminant estimates on a training set. This leads to
further improvements in accuracy, most significantly in the tail of the error
distribution.
