  Achieving safe and reliable autonomous driving relies greatly on the ability
to achieve an accurate and robust perception system; however, this cannot be
fully realized without precisely calibrated sensors. Environmental and
operational conditions as well as improper maintenance can produce calibration
errors inhibiting sensor fusion and, consequently, degrading the perception
performance. Traditionally, sensor calibration is performed in a controlled
environment with one or more known targets. Such a procedure can only be
carried out in between drives and requires manual operation; a tedious task if
needed to be conducted on a regular basis. This sparked a recent interest in
online targetless methods, capable of yielding a set of geometric
transformations based on perceived environmental features, however, the
required redundancy in sensing modalities makes this task even more
challenging, as the features captured by each modality and their
distinctiveness may vary. We present a holistic approach to performing joint
calibration of a camera-lidar-radar trio. Leveraging prior knowledge and
physical properties of these sensing modalities together with semantic
information, we propose two targetless calibration methods within a cost
minimization framework once via direct online optimization, and second via
self-supervised learning (SSL).
