  Long-term visual localization is the problem of estimating the camera pose of
a given query image in a scene whose appearance changes over time. It is an
important problem in practice, for example, encountered in autonomous driving.
In order to gain robustness to such changes, long-term localization approaches
often use segmantic segmentations as an invariant scene representation, as the
semantic meaning of each scene part should not be affected by seasonal and
other changes. However, these representations are typically not very
discriminative due to the limited number of available classes. In this paper,
we propose a new neural network, the Fine-Grained Segmentation Network (FGSN),
that can be used to provide image segmentations with a larger number of labels
and can be trained in a self-supervised fashion. In addition, we show how FGSNs
can be trained to output consistent labels across seasonal changes. We
demonstrate through extensive experiments that integrating the fine-grained
segmentations produced by our FGSNs into existing localization algorithms leads
to substantial improvements in localization performance.
