  Object classification is one of the many holy grails in computer vision and
as such has resulted in a very large number of algorithms being proposed
already. Specifically in recent years there has been considerable progress in
this area primarily due to the increased efficiency and accessibility of deep
learning techniques. In fact, for single-label object classification [i.e. only
one object present in the image] the state-of-the-art techniques employ deep
neural networks and are reporting very close to human-like performance. There
are specialized applications in which single-label object-level classification
will not suffice; for example in cases where the image contains multiple
intertwined objects of different labels.
  In this paper, we address the complex problem of multi-label pixelwise
classification. We present our distinct solution based on a convolutional
neural network (CNN) for performing multi-label pixelwise classification and
its application to large-scale urban reconstruction. A supervised learning
approach is followed for training a 13-layer CNN using both LiDAR and satellite
images. An empirical study has been conducted to determine the hyperparameters
which result in the optimal performance of the CNN. Scale invariance is
introduced by training the network on five different scales of the input and
labeled data. This results in six pixelwise classifications for each different
scale. An SVM is then trained to map the six pixelwise classifications into a
single-label. Lastly, we refine boundary pixel labels using graph-cuts for
maximum a-posteriori (MAP) estimation with Markov Random Field (MRF) priors.
The resulting pixelwise classification is then used to accurately extract and
reconstruct the buildings in large-scale urban areas. The proposed approach has
been extensively tested and the results are reported.
