  Estimating the 3D pose of a hand from a 2D image is a well-studied problem
and a requirement for several real-life applications such as virtual reality,
augmented reality, and hand gesture recognition. Currently, reasonable
estimations can be computed from single RGB images, especially when a
multi-task learning approach is used to force the system to consider the shape
of the hand when its pose is determined. However, depending on the method used
to represent the hand, the performance can drop considerably in real-life
tasks, suggesting that stable descriptions are required to achieve satisfactory
results. In this paper, we present a keypoint-based end-to-end framework for 3D
hand and pose estimation and successfully apply it to the task of hand gesture
recognition as a study case. Specifically, after a pre-processing step in which
the images are normalized, the proposed pipeline uses a multi-task semantic
feature extractor generating 2D heatmaps and hand silhouettes from RGB images,
a viewpoint encoder to predict the hand and camera view parameters, a stable
hand estimator to produce the 3D hand pose and shape, and a loss function to
guide all of the components jointly during the learning phase. Tests were
performed on a 3D pose and shape estimation benchmark dataset to assess the
proposed framework, which obtained state-of-the-art performance. Our system was
also evaluated on two hand-gesture recognition benchmark datasets and
significantly outperformed other keypoint-based approaches, indicating that it
is an effective solution that is able to generate stable 3D estimates for hand
pose and shape.
