  There have been numerous recently proposed methods for monocular depth
prediction (MDP) coupled with the equally rapid evolution of benchmarking
tools. However, we argue that MDP is currently witnessing benchmark
over-fitting and relying on metrics that are only partially helpful to gauge
the usefulness of the predictions for 3D applications. This limits the design
and development of novel methods that are truly aware of - and improving
towards estimating - the 3D structure of the scene rather than optimizing
2D-based distances. In this work, we aim to bring structural awareness to MDP,
an inherently 3D task, by exhibiting the limits of evaluation metrics towards
assessing the quality of the 3D geometry. We propose a set of metrics well
suited to evaluate the 3D geometry of MDP approaches and a novel indoor
benchmark, RIO-D3D, crucial for the proposed evaluation methodology. Our
benchmark is based on a real-world dataset featuring high-quality rendered
depth maps obtained from RGB-D reconstructions. We further demonstrate this to
help benchmark the closely-tied task of 3D scene completion.
