  We propose PR-RRN, a novel neural-network based method for Non-rigid
Structure-from-Motion (NRSfM). PR-RRN consists of Residual-Recursive Networks
(RRN) and two extra regularization losses. RRN is designed to effectively
recover 3D shape and camera from 2D keypoints with novel residual-recursive
structure. As NRSfM is a highly under-constrained problem, we propose two new
pairwise regularization to further regularize the reconstruction. The
Rigidity-based Pairwise Contrastive Loss regularizes the shape representation
by encouraging higher similarity between the representations of high-rigidity
pairs of frames than low-rigidity pairs. We propose minimum singular-value
ratio to measure the pairwise rigidity. The Pairwise Consistency Loss enforces
the reconstruction to be consistent when the estimated shapes and cameras are
exchanged between pairs. Our approach achieves state-of-the-art performance on
CMU MOCAP and PASCAL3D+ dataset.
