  People with Visual Impairments (PVI) typically recognize objects through
haptic perception. Knowing objects and materials before touching is desired by
the target users but under-explored in the field of human-centered robotics. To
fill this gap, in this work, a wearable vision-based robotic system, MateRobot,
is established for PVI to recognize materials and object categories beforehand.
To address the computational constraints of mobile platforms, we propose a
lightweight yet accurate model MateViT to perform pixel-wise semantic
segmentation, simultaneously recognizing both objects and materials. Our
methods achieve respective 40.2% and 51.1% of mIoU on COCOStuff-10K and DMS
datasets, surpassing the previous method with +5.7% and +7.0% gains. Moreover,
on the field test with participants, our wearable system reaches a score of 28
in the NASA-Task Load Index, indicating low cognitive demands and ease of use.
Our MateRobot demonstrates the feasibility of recognizing material property
through visual cues and offers a promising step towards improving the
functionality of wearable robots for PVI. The source code has been made
publicly available at
https://junweizheng93.github.io/publications/MATERobot/MATERobot.html.
