  Data-driven depth estimation methods struggle with the generalization outside
their training scenes due to the immense variability of the real-world scenes.
This problem can be partially addressed by utilising synthetically generated
images, but closing the synthetic-real domain gap is far from trivial. In this
paper, we tackle this issue by using domain invariant defocus blur as direct
supervision. We leverage defocus cues by using a permutation invariant
convolutional neural network that encourages the network to learn from the
differences between images with a different point of focus. Our proposed
network uses the defocus map as an intermediate supervisory signal. We are able
to train our model completely on synthetic data and directly apply it to a wide
range of real-world images. We evaluate our model on synthetic and real
datasets, showing compelling generalization results and state-of-the-art depth
prediction.
