  Passive, compact, single-shot 3D sensing is useful in many application areas
such as microscopy, medical imaging, surgical navigation, and autonomous
driving where form factor, time, and power constraints can exist. Obtaining
RGB-D scene information over a short imaging distance, in an ultra-compact form
factor, and in a passive, snapshot manner is challenging. Dual-pixel (DP)
sensors are a potential solution to achieve the same. DP sensors collect light
rays from two different halves of the lens in two interleaved pixel arrays,
thus capturing two slightly different views of the scene, like a stereo camera
system. However, imaging with a DP sensor implies that the defocus blur size is
directly proportional to the disparity seen between the views. This creates a
trade-off between disparity estimation vs. deblurring accuracy. To improve this
trade-off effect, we propose CADS (Coded Aperture Dual-Pixel Sensing), in which
we use a coded aperture in the imaging lens along with a DP sensor. In our
approach, we jointly learn an optimal coded pattern and the reconstruction
algorithm in an end-to-end optimization setting. Our resulting CADS imaging
system demonstrates improvement of >1.5dB PSNR in all-in-focus (AIF) estimates
and 5-6% in depth estimation quality over naive DP sensing for a wide range of
aperture settings. Furthermore, we build the proposed CADS prototypes for DSLR
photography settings and in an endoscope and a dermoscope form factor. Our
novel coded dual-pixel sensing approach demonstrates accurate RGB-D
reconstruction results in simulations and real-world experiments in a passive,
snapshot, and compact manner.
