  We consider the problem of reconstructing a dynamic scene observed from a
stereo camera. Most existing methods for depth from stereo treat different
stereo frames independently, leading to temporally inconsistent depth
predictions. Temporal consistency is especially important for immersive AR or
VR scenarios, where flickering greatly diminishes the user experience. We
propose DynamicStereo, a novel transformer-based architecture to estimate
disparity for stereo videos. The network learns to pool information from
neighboring frames to improve the temporal consistency of its predictions. Our
architecture is designed to process stereo videos efficiently through divided
attention layers. We also introduce Dynamic Replica, a new benchmark dataset
containing synthetic videos of people and animals in scanned environments,
which provides complementary training and evaluation data for dynamic stereo
closer to real applications than existing datasets. Training with this dataset
further improves the quality of predictions of our proposed DynamicStereo as
well as prior methods. Finally, it acts as a benchmark for consistent stereo
methods.
