  City models and height maps of urban areas serve as a valuable data source
for numerous applications, such as disaster management or city planning. While
this information is not globally available, it can be substituted by digital
surface models (DSMs), automatically produced from inexpensive satellite
imagery. However, stereo DSMs often suffer from noise and blur. Furthermore,
they are heavily distorted by vegetation, which is of lesser relevance for most
applications. Such basic models can be filtered by convolutional neural
networks (CNNs), trained on labels derived from digital elevation models (DEMs)
and 3D city models, in order to obtain a refined DSM. We propose a modular
multi-task learning concept that consolidates existing approaches into a
generalized framework. Our encoder-decoder models with shared encoders and
multiple task-specific decoders leverage roof type classification as a
secondary task and multiple objectives including a conditional adversarial
term. The contributing single-objective losses are automatically weighted in
the final multi-task loss function based on learned uncertainty estimates. We
evaluated the performance of specific instances of this family of network
architectures. Our method consistently outperforms the state of the art on
common data, both quantitatively and qualitatively, and generalizes well to a
new dataset of an independent study area.
