  Monocular (relative or metric) depth estimation is a critical task for
various applications, such as autonomous vehicles, augmented reality and image
editing. In recent years, with the increasing availability of mobile devices,
accurate and mobile-friendly depth models have gained importance. Increasingly
accurate models typically require more computational resources, which inhibits
the use of such models on mobile devices. The mobile use case is arguably the
most unrestricted one, which requires highly accurate yet mobile-friendly
architectures. Therefore, we try to answer the following question: How can we
improve a model without adding further complexity (i.e. parameters)? Towards
this end, we systematically explore the design space of a relative depth
estimation model from various dimensions and we show, with key design choices
and ablation studies, even an existing architecture can reach highly
competitive performance to the state of the art, with a fraction of the
complexity. Our study spans an in-depth backbone model selection process,
knowledge distillation, intermediate predictions, model pruning and loss
rebalancing. We show that our model, using only DIW as the supervisory dataset,
achieves 0.1156 WHDR on DIW with 2.6M parameters and reaches 37 FPS on a mobile
GPU, without pruning or hardware-specific optimization. A pruned version of our
model achieves 0.1208 WHDR on DIW with 1M parameters and reaches 44 FPS on a
mobile GPU.
