  We aim to perform sound event localization and detection (SELD) using
wearable equipment for a moving human, such as a pedestrian. Conventional SELD
tasks have dealt only with microphone arrays located in static positions.
However, self-motion with three rotational and three translational degrees of
freedom (6DoF) shall be considered for wearable microphone arrays. A system
trained only with a dataset using microphone arrays in a fixed position would
be unable to adapt to the fast relative motion of sound events associated with
self-motion, resulting in the degradation of SELD performance. To address this,
we designed 6DoF SELD Dataset for wearable systems, the first SELD dataset
considering the self-motion of microphones. Furthermore, we proposed a
multi-modal SELD system that jointly utilizes audio and motion tracking sensor
signals. These sensor signals are expected to help the system find useful
acoustic cues for SELD on the basis of the current self-motion state.
Experimental results on our dataset show that the proposed method effectively
improves SELD performance with a mechanism to extract acoustic features
conditioned by sensor signals.
