  Active stereo cameras that recover depth from structured light captures have
become a cornerstone sensor modality for 3D scene reconstruction and
understanding tasks across application domains. Existing active stereo cameras
project a pseudo-random dot pattern on object surfaces to extract disparity
independently of object texture. Such hand-crafted patterns are designed in
isolation from the scene statistics, ambient illumination conditions, and the
reconstruction method. In this work, we propose the first method to jointly
learn structured illumination and reconstruction, parameterized by a
diffractive optical element and a neural network, in an end-to-end fashion. To
this end, we introduce a novel differentiable image formation model for active
stereo, relying on both wave and geometric optics, and a novel trinocular
reconstruction network. The jointly optimized pattern, which we dub "Polka
Lines," together with the reconstruction network, achieve state-of-the-art
active-stereo depth estimates across imaging conditions. We validate the
proposed method in simulation and on a hardware prototype, and show that our
method outperforms existing active stereo systems.
