  Humans rely on properties of the materials that make up objects to guide our
interactions with them. Grasping smooth materials, for example, requires care,
and softness is an ideal property for fabric used in bedding. Even when these
properties are not visual (e.g. softness is a physical property), we may still
infer their presence visually. We refer to such material properties as visual
material attributes. Recognizing these attributes in images can contribute
valuable information for general scene understanding and material recognition.
Unlike well-known object and scene attributes, visual material attributes are
local properties with no fixed shape or spatial extent. We show that given a
set of images annotated with known material attributes, we may accurately
recognize the attributes from small local image patches. Obtaining such
annotations in a consistent fashion at scale, however, is challenging. To
address this, we introduce a method that allows us to probe the human visual
perception of materials by asking simple yes/no questions comparing pairs of
image patches. This provides sufficient weak supervision to build a set of
attributes and associated classifiers that, while unnamed, serve the same
function as the named attributes we use to describe materials. Doing so allows
us to recognize visual material attributes without resorting to exhaustive
manual annotation of a fixed set of named attributes. Furthermore, we show that
this method may be integrated in the end-to-end learning of a material
classification CNN to simultaneously recognize materials and discover their
visual attributes. Our experimental results show that visual material
attributes, whether named or automatically discovered, provide a useful
intermediate representation for known material categories themselves as well as
a basis for transfer learning when recognizing previously-unseen categories.
