  Tactile sensing plays an irreplaceable role in robotic material recognition.
It enables robots to distinguish material properties such as their local
geometry and textures, especially for materials like textiles. However, most
tactile recognition methods can only classify known materials that have been
touched and trained with tactile data, yet cannot classify unknown materials
that are not trained with tactile data. To solve this problem, we propose a
tactile zero-shot learning framework to recognise unknown materials when they
are touched for the first time without requiring training tactile samples. The
visual modality, providing tactile cues from sight, and semantic attributes,
giving high-level characteristics, are combined together to bridge the gap
between touched classes and untouched classes. A generative model is learnt to
synthesise tactile features according to corresponding visual images and
semantic embeddings, and then a classifier can be trained using the synthesised
tactile features of untouched materials for zero-shot recognition. Extensive
experiments demonstrate that our proposed multimodal generative model can
achieve a high recognition accuracy of 83.06% in classifying materials that
were not touched before. The robotic experiment demo and the dataset are
available at https://sites.google.com/view/multimodalzsl.
