  Estimating 3D hand meshes from RGB images robustly is a highly desirable
task, made challenging due to the numerous degrees of freedom, and issues such
as self similarity and occlusions. Previous methods generally either use
parametric 3D hand models or follow a model-free approach. While the former can
be considered more robust, e.g. to occlusions, they are less expressive. We
propose a hybrid approach, utilizing a deep neural network and differential
rendering based optimization to demonstrably achieve the best of both worlds.
In addition, we explore Virtual Reality (VR) as an application. Most VR
headsets are nowadays equipped with multiple cameras, which we can leverage by
extending our method to the egocentric stereo domain. This extension proves to
be more resilient to the above mentioned issues. Finally, as a use-case, we
show that the improved image-model alignment can be used to acquire the user's
hand texture, which leads to a more realistic virtual hand representation.
