  Active speaker detection is an important component in video analysis
algorithms for applications such as speaker diarization, video re-targeting for
meetings, speech enhancement, and human-robot interaction. The absence of a
large, carefully labeled audio-visual dataset for this task has constrained
algorithm evaluations with respect to data diversity, environments, and
accuracy. This has made comparisons and improvements difficult. In this paper,
we present the AVA Active Speaker detection dataset (AVA-ActiveSpeaker) that
will be released publicly to facilitate algorithm development and enable
comparisons. The dataset contains temporally labeled face tracks in video,
where each face instance is labeled as speaking or not, and whether the speech
is audible. This dataset contains about 3.65 million human labeled frames or
about 38.5 hours of face tracks, and the corresponding audio. We also present a
new audio-visual approach for active speaker detection, and analyze its
performance, demonstrating both its strength and the contributions of the
dataset.
