  We propose a real-time vision-based teleoperation approach for robotic arms
that employs a single depth-based camera, exempting the user from the need for
any wearable devices. By employing a natural user interface, this novel
approach leverages the conventional fine-tuning control, turning it into a
direct body pose capture process. The proposed approach is comprised of two
main parts. The first is a nonlinear customizable pose mapping based on
Thin-Plate Splines (TPS), to directly transfer human body motion to robotic arm
motion in a nonlinear fashion, thus allowing matching dissimilar bodies with
different workspace shapes and kinematic constraints. The second is a Deep
Neural Network hand-state classifier based on Long-term Recurrent Convolutional
Networks (LRCN) that exploits the temporal coherence of the acquired depth
data. We validate, evaluate and compare our approach through both classical
cross-validation experiments of the proposed hand state classifier; and user
studies over a set of practical experiments involving variants of
pick-and-place and manufacturing tasks. Results revealed that LRCN networks
outperform single image Convolutional Neural Networks; and that users' learning
curves were steep, thus allowing the successful completion of the proposed
tasks. When compared to a previous approach, the TPS approach revealed no
increase in task complexity and similar times of completion, while providing
more precise operation in regions closer to workspace boundaries.
