  Active speaker detection in videos addresses associating a source face,
visible in the video frames, with the underlying speech in the audio modality.
The two primary sources of information to derive such a speech-face
relationship are i) visual activity and its interaction with the speech signal
and ii) co-occurrences of speakers' identities across modalities in the form of
face and speech. The two approaches have their limitations: the audio-visual
activity models get confused with other frequently occurring vocal activities,
such as laughing and chewing, while the speakers' identity-based methods are
limited to videos having enough disambiguating information to establish a
speech-face association. Since the two approaches are independent, we
investigate their complementary nature in this work. We propose a novel
unsupervised framework to guide the speakers' cross-modal identity association
with the audio-visual activity for active speaker detection. Through
experiments on entertainment media videos from two benchmark datasets, the AVA
active speaker (movies) and Visual Person Clustering Dataset (TV shows), we
show that a simple late fusion of the two approaches enhances the active
speaker detection performance.
