  The study of dexterous manipulation has provided important insights in humans
sensorimotor control as well as inspiration for manipulation strategies in
robotic hands. Previous work focused on experimental environment with
restrictions. Here we describe a method using the deformation and color
distribution of the fingernail and its surrounding skin, to estimate the
fingertip forces, torques and contact surface curvatures for various objects,
including the shape and material of the contact surfaces and the weight of the
objects. The proposed method circumvents limitations associated with sensorized
objects, gloves or fixed contact surface type. In addition, compared with
previous single finger estimation in an experimental environment, we extend the
approach to multiple finger force estimation, which can be used for
applications such as human grasping analysis. Four algorithms are used, c.q.,
Gaussian process (GP), Convolutional Neural Networks (CNN), Neural Networks
with Fast Dropout (NN-FD) and Recurrent Neural Networks with Fast Dropout
(RNN-FD), to model a mapping from images to the corresponding labels. The
results further show that the proposed method has high accuracy to predict
force, torque and contact surface.
