  In this paper, we propose Phoneme Discretized Saliency Maps (PDSM), a
discretization algorithm for saliency maps that takes advantage of phoneme
boundaries for explainable detection of AI-generated voice. We experimentally
show with two different Text-to-Speech systems (i.e., Tacotron2 and
Fastspeech2) that the proposed algorithm produces saliency maps that result in
more faithful explanations compared to standard posthoc explanation methods.
Moreover, by associating the saliency maps to the phoneme representations, this
methodology generates explanations that tend to be more understandable than
standard saliency maps on magnitude spectrograms.
