  Augmented Reality (AR) devices, emerging as prominent mobile interaction
platforms, face challenges in user safety, particularly concerning oncoming
vehicles. While some solutions leverage onboard camera arrays, these cameras
often have limited field-of-view (FoV) with front or downward perspectives.
Addressing this, we propose a new out-of-view semantic segmentation task and
Segment Beyond View (SBV), a novel audio-visual semantic segmentation method.
SBV supplements the visual modality, which miss the information beyond FoV,
with the auditory information using a teacher-student distillation model
(Omni2Ego). The model consists of a vision teacher utilising panoramic
information, an auditory teacher with 8-channel audio, and an audio-visual
student that takes views with limited FoV and binaural audio as input and
produce semantic segmentation for objects outside FoV. SBV outperforms existing
models in comparative evaluations and shows a consistent performance across
varying FoV ranges and in monaural audio settings.
