  Recent deep learning approaches for multi-view depth estimation are employed
either in a depth-from-video or a multi-view stereo setting. Despite different
settings, these approaches are technically similar: they correlate multiple
source views with a keyview to estimate a depth map for the keyview. In this
work, we introduce the Robust Multi-View Depth Benchmark that is built upon a
set of public datasets and allows evaluation in both settings on data from
different domains. We evaluate recent approaches and find imbalanced
performances across domains. Further, we consider a third setting, where camera
poses are available and the objective is to estimate the corresponding depth
maps with their correct scale. We show that recent approaches do not generalize
across datasets in this setting. This is because their cost volume output runs
out of distribution. To resolve this, we present the Robust MVD Baseline model
for multi-view depth estimation, which is built upon existing components but
employs a novel scale augmentation procedure. It can be applied for robust
multi-view depth estimation, independent of the target data. We provide code
for the proposed benchmark and baseline model at
https://github.com/lmb-freiburg/robustmvd.
