  Visual recognition of materials and their states is essential for
understanding most aspects of the world, from determining whether food is
cooked, metal is rusted, or a chemical reaction has occurred. However, current
image recognition methods are limited to specific classes and properties and
can't handle the vast number of material states in the world. To address this,
we present MatSim: the first dataset and benchmark for computer vision-based
recognition of similarities and transitions between materials and textures,
focusing on identifying any material under any conditions using one or a few
examples. The dataset contains synthetic and natural images. The synthetic
images were rendered using giant collections of textures, objects, and
environments generated by computer graphics artists. We use mixtures and
gradual transitions between materials to allow the system to learn cases with
smooth transitions between states (like gradually cooked food). We also render
images with materials inside transparent containers to support beverage and
chemistry lab use cases. We use this dataset to train a siamese net that
identifies the same material in different objects, mixtures, and environments.
The descriptor generated by this net can be used to identify the states of
materials and their subclasses using a single image. We also present the first
few-shot material recognition benchmark with images from a wide range of
fields, including the state of foods and drinks, types of grounds, and many
other use cases. We show that a net trained on the MatSim synthetic dataset
outperforms state-of-the-art models like Clip on the benchmark and also
achieves good results on other unsupervised material classification tasks.
