  With the recent advances in voice synthesis, AI-synthesized fake voices are
indistinguishable to human ears and widely are applied to produce realistic and
natural DeepFakes, exhibiting real threats to our society. However, effective
and robust detectors for synthesized fake voices are still in their infancy and
are not ready to fully tackle this emerging threat. In this paper, we devise a
novel approach, named \emph{DeepSonar}, based on monitoring neuron behaviors of
speaker recognition (SR) system, \ie, a deep neural network (DNN), to discern
AI-synthesized fake voices. Layer-wise neuron behaviors provide an important
insight to meticulously catch the differences among inputs, which are widely
employed for building safety, robust, and interpretable DNNs. In this work, we
leverage the power of layer-wise neuron activation patterns with a conjecture
that they can capture the subtle differences between real and AI-synthesized
fake voices, in providing a cleaner signal to classifiers than raw inputs.
Experiments are conducted on three datasets (including commercial products from
Google, Baidu, \etc) containing both English and Chinese languages to
corroborate the high detection rates (98.1\% average accuracy) and low false
alarm rates (about 2\% error rate) of DeepSonar in discerning fake voices.
Furthermore, extensive experimental results also demonstrate its robustness
against manipulation attacks (\eg, voice conversion and additive real-world
noises). Our work further poses a new insight into adopting neuron behaviors
for effective and robust AI aided multimedia fakes forensics as an inside-out
approach instead of being motivated and swayed by various artifacts introduced
in synthesizing fakes.
