  While learning based depth estimation from images/videos has achieved
substantial progress, there still exist intrinsic limitations. Supervised
methods are limited by a small amount of ground truth or labeled data and
unsupervised methods for monocular videos are mostly based on the static scene
assumption, not performing well on real world scenarios with the presence of
dynamic objects. In this paper, we propose a new learning based method
consisting of DepthNet, PoseNet and Region Deformer Networks (RDN) to estimate
depth from unconstrained monocular videos without ground truth supervision. The
core contribution lies in RDN for proper handling of rigid and non-rigid
motions of various objects such as rigidly moving cars and deformable humans.
In particular, a deformation based motion representation is proposed to model
individual object motion on 2D images. This representation enables our method
to be applicable to diverse unconstrained monocular videos. Our method can not
only achieve the state-of-the-art results on standard benchmarks KITTI and
Cityscapes, but also show promising results on a crowded pedestrian tracking
dataset, which demonstrates the effectiveness of the deformation based motion
representation. Code and trained models are available at
https://github.com/haofeixu/rdn4depth.
