  Previous learning based hand pose estimation methods does not fully exploit
the prior information in hand model geometry. Instead, they usually rely a
separate model fitting step to generate valid hand poses. Such a post
processing is inconvenient and sub-optimal. In this work, we propose a model
based deep learning approach that adopts a forward kinematics based layer to
ensure the geometric validity of estimated poses. For the first time, we show
that embedding such a non-linear generative process in deep learning is
feasible for hand pose estimation. Our approach is verified on challenging
public datasets and achieves state-of-the-art performance.
