  Modern optical satellite sensors enable high-resolution stereo reconstruction
from space. But the challenging imaging conditions when observing the Earth
from space push stereo matching to its limits. In practice, the resulting
digital surface models (DSMs) are fairly noisy and often do not attain the
accuracy needed for high-resolution applications such as 3D city modeling.
Arguably, stereo correspondence based on low-level image similarity is
insufficient and should be complemented with a-priori knowledge about the
expected surface geometry beyond basic local smoothness. To that end, we
introduce ResDepth, a convolutional neural network that learns such an
expressive geometric prior from example data. ResDepth refines an initial, raw
stereo DSM while conditioning the refinement on the images. I.e., it acts as a
smart, learned post-processing filter and can seamlessly complement any stereo
matching pipeline. In a series of experiments, we find that the proposed method
consistently improves stereo DSMs both quantitatively and qualitatively. We
show that the prior encoded in the network weights captures meaningful
geometric characteristics of urban design, which also generalize across
different districts and even from one city to another. Moreover, we demonstrate
that, by training on a variety of stereo pairs, ResDepth can acquire a
sufficient degree of invariance against variations in imaging conditions and
acquisition geometry.
