  In this letter, we propose a pseudo-siamese convolutional neural network
(CNN) architecture that enables to solve the task of identifying corresponding
patches in very-high-resolution (VHR) optical and synthetic aperture radar
(SAR) remote sensing imagery. Using eight convolutional layers each in two
parallel network streams, a fully connected layer for the fusion of the
features learned in each stream, and a loss function based on binary
cross-entropy, we achieve a one-hot indication if two patches correspond or
not. The network is trained and tested on an automatically generated dataset
that is based on a deterministic alignment of SAR and optical imagery via
previously reconstructed and subsequently co-registered 3D point clouds. The
satellite images, from which the patches comprising our dataset are extracted,
show a complex urban scene containing many elevated objects (i.e. buildings),
thus providing one of the most difficult experimental environments. The
achieved results show that the network is able to predict corresponding patches
with high accuracy, thus indicating great potential for further development
towards a generalized multi-sensor key-point matching procedure. Index
Terms-synthetic aperture radar (SAR), optical imagery, data fusion, deep
learning, convolutional neural networks (CNN), image matching, deep matching
