  In this paper, we develop a high-dimensional map building technique that
incorporates raw pixelated semantic measurements into the map representation.
The proposed technique uses Gaussian Processes (GPs) multi-class classification
for map inference and is the natural extension of GP occupancy maps from binary
to multi-class form. The technique exploits the continuous property of GPs and,
as a result, the map can be inferred with any resolution. In addition, the
proposed GP Semantic Map (GPSM) learns the structural and semantic correlation
from measurements rather than resorting to assumptions, and can flexibly learn
the spatial correlation as well as any additional non-spatial correlation
between map points. We extend the OctoMap to Semantic OctoMap representation
and compare with the GPSM mapping performance using NYU Depth V2 dataset.
Evaluations of the proposed technique on multiple partially labeled RGBD scans
and labels from noisy image segmentation show that the GP semantic map can
handle sparse measurements, missing labels in the point cloud, as well as noise
corrupted labels.
