  Monocular depth estimation and defocus estimation are two fundamental tasks
in computer vision. Most existing methods treat depth estimation and defocus
estimation as two separate tasks, ignoring the strong connection between them.
In this work, we propose a multi-task learning network consisting of an encoder
with two decoders to estimate the depth and defocus map from a single focused
image. Through the multi-task network, the depth estimation facilitates the
defocus estimation to get better results in the weak texture region and the
defocus estimation facilitates the depth estimation by the strong physical
connection between the two maps. We set up a dataset (named ALL-in-3D dataset)
which is the first all-real image dataset consisting of 100K sets of
all-in-focus images, focused images with focus depth, depth maps, and defocus
maps. It enables the network to learn features and solid physical connections
between the depth and real defocus images. Experiments demonstrate that the
network learns more solid features from the real focused images than the
synthetic focused images. Benefiting from this multi-task structure where
different tasks facilitate each other, our depth and defocus estimations
achieve significantly better performance than other state-of-art algorithms.
The code and dataset will be publicly available at
https://github.com/cubhe/MDDNet.
