  We study the problem of estimating the relative depth order of point pairs in
a monocular image. Recent advances mainly focus on using deep convolutional
neural networks (DCNNs) to learn and infer the ordinal information from
multiple contextual information of the points pair such as global scene
context, local contextual information, and the locations. However, it remains
unclear how much each context contributes to the task. To address this, we
first examine the contribution of each context cue [1], [2] to the performance
in the context of depth order estimation. We find out the local context
surrounding the points pair contributes the most and the global scene context
helps little. Based on the findings, we propose a simple method, using a
multi-scale densely-connected network to tackle the task. Instead of learning
the global structure, we dedicate to explore the local structure by learning to
regress from regions of multiple sizes around the point pairs. Moreover, we use
the recent densely connected network [3] to encourage substantial feature reuse
as well as deepen our network to boost the performance. We show in experiments
that the results of our approach is on par with or better than the
state-of-the-art methods with the benefit of using only a small number of
training data.
