  In this paper, based on the assumption that the object boundaries (e.g.,
buildings) from the over-view data should coincide with footprints of
fa\c{c}ade 3D points generated from street-view photogrammetric images, we aim
to address this problem by proposing a fully automated geo-registration method
for cross-view data, which utilizes semantically segmented object boundaries as
view-invariant features under a global optimization framework through
graph-matching: taking the over-view point clouds generated from
stereo/multi-stereo satellite images and the street-view point clouds generated
from monocular video images as the inputs, the proposed method models segments
of buildings as nodes of graphs, both detected from the satellite-based and
street-view based point clouds, thus to form the registration as a
graph-matching problem to allow non-rigid matches; to enable a robust solution
and fully utilize the topological relations between these segments, we propose
to address the graph-matching problem on its conjugate graph solved through a
belief-propagation algorithm. The matched nodes will be subject to a further
optimization to allow precise-registration, followed by a constrained bundle
adjustment on the street-view image to keep 2D29 3D consistencies, which yields
well-registered street-view images and point clouds to the satellite point
clouds.
