  Observing and recognizing materials is a fundamental part of our daily life.
Under typical viewing conditions, we are capable of effortlessly identifying
the objects that surround us and recognizing the materials they are made of.
Nevertheless, understanding the underlying perceptual processes that take place
to accurately discern the visual properties of an object is a long-standing
problem. In this work, we perform a comprehensive and systematic analysis of
how the interplay of geometry, illumination, and their spatial frequencies
affects human performance on material recognition tasks. We carry out
large-scale behavioral experiments where participants are asked to recognize
different reference materials among a pool of candidate samples. In the
different experiments, we carefully sample the information in the frequency
domain of the stimuli. From our analysis, we find significant first-order
interactions between the geometry and the illumination, of both the reference
and the candidates. In addition, we observe that simple image statistics and
higher-order image histograms do not correlate with human performance.
Therefore, we perform a high-level comparison of highly non-linear statistics
by training a deep neural network on material recognition tasks. Our results
show that such models can accurately classify materials, which suggests that
they are capable of defining a meaningful representation of material appearance
from labeled proximal image data. Last, we find preliminary evidence that these
highly non-linear models and humans may use similar high-level factors for
material recognition tasks.
