  In this paper, a self-supervised model that simultaneously predicts a
sequence of future frames from video-input with a novel spatial-temporal
attention (ST) network is proposed. The ST transformer network allows
constraining both temporal consistency across future frames whilst constraining
consistency across spatial objects in the image at different scales. This was
not the case in prior works for depth prediction, which focused on predicting a
single frame as output. The proposed model leverages prior scene knowledge such
as object shape and texture similar to single-image depth inference methods,
whilst also constraining the motion and geometry from a sequence of input
images. Apart from the transformer architecture, one of the main contributions
with respect to prior works lies in the objective function that enforces
spatio-temporal consistency across a sequence of output frames rather than a
single output frame. As will be shown, this results in more accurate and robust
depth sequence forecasting. The model achieves highly accurate depth
forecasting results that outperform existing baselines on the KITTI benchmark.
Extensive ablation studies were performed to assess the effectiveness of the
proposed techniques. One remarkable result of the proposed model is that it is
implicitly capable of forecasting the motion of objects in the scene, rather
than requiring complex models involving multi-object detection, segmentation
and tracking.
