  Nowadays, Earth Observation systems provide a multitude of heterogeneous
remote sensing data. How to manage such richness leveraging its complementarity
is a crucial chal- lenge in modern remote sensing analysis. Data Fusion
techniques deal with this point proposing method to combine and exploit
complementarity among the different data sensors. Considering optical Very High
Spatial Resolution (VHSR) images, satellites obtain both Multi Spectral (MS)
and panchro- matic (PAN) images at different spatial resolution. VHSR images
are extensively exploited to produce land cover maps to deal with agricultural,
ecological, and socioeconomic issues as well as assessing ecosystem status,
monitoring biodiversity and provid- ing inputs to conceive food risk monitoring
systems. Common techniques to produce land cover maps from such VHSR images
typically opt for a prior pansharpening of the multi-resolution source for a
full resolution processing. Here, we propose a new deep learning architecture
to jointly use PAN and MS imagery for a direct classification without any prior
image fusion or resampling process. By managing the spectral information at its
native spatial resolution, our method, named MRFusion, aims at avoiding the
possible infor- mation loss induced by pansharpening or any other hand-crafted
preprocessing. Moreover, the proposed architecture is suitably designed to
learn non-linear transformations of the sources with the explicit aim of taking
as much as possible advantage of the complementarity of PAN and MS imagery.
Experiments are carried out on two-real world scenarios depicting large areas
with different land cover characteristics. The characteristics of the proposed
scenarios underline the applicability and the generality of our method in
operational settings.
