  Autonomous cars need continuously updated depth information. Thus far, depth
is mostly estimated independently for a single frame at a time, even if the
method starts from video input. Our method produces a time series of depth
maps, which makes it an ideal candidate for online learning approaches. In
particular, we put three different types of depth estimation (supervised depth
prediction, self-supervised depth prediction, and self-supervised depth
completion) into a common framework. We integrate the corresponding networks
with a ConvLSTM such that the spatiotemporal structures of depth across frames
can be exploited to yield a more accurate depth estimation. Our method is
flexible. It can be applied to monocular videos only or be combined with
different types of sparse depth patterns. We carefully study the architecture
of the recurrent network and its training strategy. We are first to
successfully exploit recurrent networks for real-time self-supervised monocular
depth estimation and completion. Extensive experiments show that our recurrent
method outperforms its image-based counterpart consistently and significantly
in both self-supervised scenarios. It also outperforms previous depth
estimation methods of the three popular groups. Please refer to
https://www.trace.ethz.ch/publications/2020/rec_depth_estimation/ for details.
