  Humans can accurately determine whether the object in hand has slipped or not
by visual and tactile perception. However, it is still a challenge for robots
to detect in-hand object slip through visuo-tactile fusion. To address this
issue, a novel visuo-tactile fusion deep neural network is proposed to detect
slip, which is a time-dependent continuous action. By using the multi-scale
temporal convolution network (MS-TCN) to extract the temporal features of
visual and tactile data, the slip can be detected effectively. In this paper, a
7-dregree-of-freedom (7-DoF) robot manipulator equipped with a camera and a
tactile sensor is used for data collection on 50 daily objects with different
shapes, materials, sizes, and weights. Therefore, a dataset is built, where the
grasping data of 40 objects and 10 objects are used for network training and
testing, respectively. The detection accuracy is 96.96% based on the proposed
model. Also, the proposed model is compared with a visuo-tactile fusion deep
neural network (DNN) based on long short-term memory network (LSTM) on the
collected dataset and a public dataset using the GelSight tactile sensor. The
results demonstrate that the proposed model performs better on both dataset.
The proposed model can help robots grasp daily objects reliably. In addition,
it can be used in grasping force control, grasping policy generation and
dexterous manipulation.
