  We present PICCOLO, a simple and efficient algorithm for omnidirectional
localization. Given a colored point cloud and a 360 panorama image of a scene,
our objective is to recover the camera pose at which the panorama image is
taken. Our pipeline works in an off-the-shelf manner with a single image given
as a query and does not require any training of neural networks or collecting
ground-truth poses of images. Instead, we match each point cloud color to the
holistic view of the panorama image with gradient-descent optimization to find
the camera pose. Our loss function, called sampling loss, is point
cloud-centric, evaluated at the projected location of every point in the point
cloud. In contrast, conventional photometric loss is image-centric, comparing
colors at each pixel location. With a simple change in the compared entities,
sampling loss effectively overcomes the severe visual distortion of
omnidirectional images, and enjoys the global context of the 360 view to handle
challenging scenarios for visual localization. PICCOLO outperforms existing
omnidirectional localization algorithms in both accuracy and stability when
evaluated in various environments. Code is available at
\url{https://github.com/82magnolia/panoramic-localization/}.
