  In this work, we present a methodology to shape a fisheye-specific
representation space that reflects the interaction between distortion and
semantic context present in this data modality. Fisheye data has the wider
field of view advantage over other types of cameras, but this comes at the
expense of high radial distortion. As a result, objects further from the center
exhibit deformations that make it difficult for a model to identify their
semantic context. While previous work has attempted architectural and training
augmentation changes to alleviate this effect, no work has attempted to guide
the model towards learning a representation space that reflects this
interaction between distortion and semantic context inherent to fisheye data.
We introduce an approach to exploit this relationship by first extracting
distortion class labels based on an object's distance from the center of the
image. We then shape a backbone's representation space with a weighted
contrastive loss that constrains objects of the same semantic class and
distortion class to be close to each other within a lower dimensional embedding
space. This backbone trained with both semantic and distortion information is
then fine-tuned within an object detection setting to empirically evaluate the
quality of the learnt representation. We show this method leads to performance
improvements by as much as 1.1% mean average precision over standard object
detection strategies and .6% improvement over other state of the art
representation learning approaches.
