  This paper reports on a novel nonparametric rigid point cloud registration
framework that jointly integrates geometric and semantic measurements such as
color or semantic labels into the alignment process and does not require
explicit data association. The point clouds are represented as nonparametric
functions in a reproducible kernel Hilbert space. The alignment problem is
formulated as maximizing the inner product between two functions, essentially a
sum of weighted kernels, each of which exploits the local geometric and
semantic features. As a result of the continuous models, analytical gradients
can be computed, and a local solution can be obtained by optimization over the
rigid body transformation group. Besides, we present a new point cloud
alignment metric that is intrinsic to the proposed framework and takes into
account geometric and semantic information. The evaluations using publicly
available stereo and RGB-D datasets show that the proposed method outperforms
state-of-the-art outdoor and indoor frame-to-frame registration methods. An
open-source GPU implementation is also provided.
