  Multisource image analysis that leverages complementary spectral, spatial,
and structural information benefits fine-grained object recognition that aims
to classify an object into one of many similar subcategories. However, for
multisource tasks that involve relatively small objects, even the smallest
registration errors can introduce high uncertainty in the classification
process. We approach this problem from a weakly supervised learning perspective
in which the input images correspond to larger neighborhoods around the
expected object locations where an object with a given class label is present
in the neighborhood without any knowledge of its exact location. The proposed
method uses a single-source deep instance attention model with parallel
branches for joint localization and classification of objects, and extends this
model into a multisource setting where a reference source that is assumed to
have no location uncertainty is used to aid the fusion of multiple sources in
four different levels: probability level, logit level, feature level, and pixel
level. We show that all levels of fusion provide higher accuracies compared to
the state-of-the-art, with the best performing method of feature-level fusion
resulting in 53% accuracy for the recognition of 40 different types of trees,
corresponding to an improvement of 5.7% over the best performing baseline when
RGB, multispectral, and LiDAR data are used. We also provide an in-depth
comparison by evaluating each model at various parameter complexity settings,
where the increased model capacity results in a further improvement of 6.3%
over the default capacity setting.
