  We present a method to infer a dense depth map from a color image and
associated sparse depth measurements. Our main contribution lies in the design
of an annealing process for determining co-visibility (occlusions,
disocclusions) and the degree of regularization to impose on the model. We show
that regularization and co-visibility are related via the fitness (residual) of
model to data and both can be unified into a single framework to improve the
learning process. Our method is an adaptive weighting scheme that guides
optimization by measuring the residual at each pixel location over each
training step for (i) estimating a soft visibility mask and (ii) determining
the amount of regularization. We demonstrate the effectiveness our method by
applying it to several recent unsupervised depth completion methods and
improving their performance on public benchmark datasets, without incurring
additional trainable parameters or increase in inference time. Code available
at: https://github.com/alexklwong/adaframe-depth-completion.
