  Tactile servoing is an important technique because it enables robots to
manipulate objects with precision and accuracy while adapting to changes in
their environments in real-time. One approach for tactile servo control with
high-resolution soft tactile sensors is to estimate the contact pose relative
to an object surface using a convolutional neural network (CNN) for use as a
feedback signal. In this paper, we investigate how the surface pose estimation
model can be extended to include shear, and utilize these combined
pose-and-shear models to develop a tactile robotic system that can be
programmed for diverse non-prehensile manipulation tasks, such as object
tracking, surface following, single-arm object pushing and dual-arm object
pushing. In doing this, two technical challenges had to be overcome. Firstly,
the use of tactile data that includes shear-induced slippage can lead to
error-prone estimates unsuitable for accurate control, and so we modified the
CNN into a Gaussian-density neural network and used a discriminative Bayesian
filter to improve the predictions with a state dynamics model that utilizes the
robot kinematics. Secondly, to achieve smooth robot motion in 3D space while
interacting with objects, we used SE(3) velocity-based servo control, which
required re-deriving the Bayesian filter update equations using Lie group
theory, as many standard assumptions do not hold for state variables defined on
non-Euclidean manifolds. In future, we believe that pose and shear-based
tactile servoing will enable many object manipulation tasks and the
fully-dexterous utilization of multi-fingered tactile robot hands. Video:
https://www.youtube.com/watch?v=xVs4hd34ek0
