  Speaker diarization may be difficult to achieve when applied to narrative
films, where speakers usually talk in adverse acoustic conditions: background
music, sound effects, wide variations in intonation may hide the inter-speaker
variability and make audio-based speaker diarization approaches error prone. On
the other hand, such fictional movies exhibit strong regularities at the image
level, particularly within dialogue scenes. In this paper, we propose to
perform speaker diarization within dialogue scenes of TV series by combining
the audio and video modalities: speaker diarization is first performed by using
each modality, the two resulting partitions of the instance set are then
optimally matched, before the remaining instances, corresponding to cases of
disagreement between both modalities, are finally processed. The results
obtained by applying such a multi-modal approach to fictional films turn out to
outperform those obtained by relying on a single modality.
