  A Colored point cloud, as a simple and efficient 3D representation, has many
advantages in various fields, including robotic navigation and scene
reconstruction. This representation is now commonly used in 3D reconstruction
tasks relying on cameras and LiDARs. However, fusing data from these two types
of sensors is poorly performed in many existing frameworks, leading to
unsatisfactory mapping results, mainly due to inaccurate camera poses. This
paper presents OmniColor, a novel and efficient algorithm to colorize point
clouds using an independent 360-degree camera. Given a LiDAR-based point cloud
and a sequence of panorama images with initial coarse camera poses, our
objective is to jointly optimize the poses of all frames for mapping images
onto geometric reconstructions. Our pipeline works in an off-the-shelf manner
that does not require any feature extraction or matching process. Instead, we
find optimal poses by directly maximizing the photometric consistency of LiDAR
maps. In experiments, we show that our method can overcome the severe visual
distortion of omnidirectional images and greatly benefit from the wide field of
view (FOV) of 360-degree cameras to reconstruct various scenarios with accuracy
and stability. The code will be released at
https://github.com/liubonan123/OmniColor/.
