  Humans naturally perceive surrounding scenes by unifying sound and sight in a
first-person view. Likewise, machines are advanced to approach human
intelligence by learning with multisensory inputs from an egocentric
perspective. In this paper, we explore the challenging egocentric audio-visual
object localization task and observe that 1) egomotion commonly exists in
first-person recordings, even within a short duration; 2) The out-of-view sound
components can be created while wearers shift their attention. To address the
first problem, we propose a geometry-aware temporal aggregation module to
handle the egomotion explicitly. The effect of egomotion is mitigated by
estimating the temporal geometry transformation and exploiting it to update
visual representations. Moreover, we propose a cascaded feature enhancement
module to tackle the second issue. It improves cross-modal localization
robustness by disentangling visually-indicated audio representation. During
training, we take advantage of the naturally available audio-visual temporal
synchronization as the ``free'' self-supervision to avoid costly labeling. We
also annotate and create the Epic Sounding Object dataset for evaluation
purposes. Extensive experiments show that our method achieves state-of-the-art
localization performance in egocentric videos and can be generalized to diverse
audio-visual scenes.
