  Depth from defocus (DfD) and stereo matching are two most studied passive
depth sensing schemes. The techniques are essentially complementary: DfD can
robustly handle repetitive textures that are problematic for stereo matching
whereas stereo matching is insensitive to defocus blurs and can handle large
depth range. In this paper, we present a unified learning-based technique to
conduct hybrid DfD and stereo matching. Our input is image triplets: a stereo
pair and a defocused image of one of the stereo views. We first apply
depth-guided light field rendering to construct a comprehensive training
dataset for such hybrid sensing setups. Next, we adopt the hourglass network
architecture to separately conduct depth inference from DfD and stereo.
Finally, we exploit different connection methods between the two separate
networks for integrating them into a unified solution to produce high fidelity
3D disparity maps. Comprehensive experiments on real and synthetic data show
that our new learning-based hybrid 3D sensing technique can significantly
improve accuracy and robustness in 3D reconstruction.
