  In this work we present a self-supervised learning framework to
simultaneously train two Convolutional Neural Networks (CNNs) to predict depth
and surface normals from a single image. In contrast to most existing
frameworks which represent outdoor scenes as fronto-parallel planes at
piece-wise smooth depth, we propose to predict depth with surface orientation
while assuming that natural scenes have piece-wise smooth normals. We show that
a simple depth-normal consistency as a soft-constraint on the predictions is
sufficient and effective for training both these networks simultaneously. The
trained normal network provides state-of-the-art predictions while the depth
network, relying on much realistic smooth normal assumption, outperforms the
traditional self-supervised depth prediction network by a large margin on the
KITTI benchmark. Demo video: https://youtu.be/ZD-ZRsw7hdM
