  Audio-visual active speaker detection (AVASD) is well-developed, and now is
an indispensable front-end for several multi-modal applications. However, to
the best of our knowledge, the adversarial robustness of AVASD models hasn't
been investigated, not to mention the effective defense against such attacks.
In this paper, we are the first to reveal the vulnerability of AVASD models
under audio-only, visual-only, and audio-visual adversarial attacks through
extensive experiments. What's more, we also propose a novel audio-visual
interaction loss (AVIL) for making attackers difficult to find feasible
adversarial examples under an allocated attack budget. The loss aims at pushing
the inter-class embeddings to be dispersed, namely non-speech and speech
clusters, sufficiently disentangled, and pulling the intra-class embeddings as
close as possible to keep them compact. Experimental results show the AVIL
outperforms the adversarial training by 33.14 mAP (%) under multi-modal
attacks.
