  We present an analysis of diacritic recognition performance in Arabic
Automatic Speech Recognition (ASR) systems. As most existing Arabic speech
corpora do not contain all diacritical marks, which represent short vowels and
other phonetic information in Arabic script, current state-of-the-art ASR
models do not produce full diacritization in their output. Automatic text-based
diacritization has previously been employed both as a pre-processing step to
train diacritized ASR, or as a post-processing step to diacritize the resulting
ASR hypotheses. It is generally believed that input diacritization degrades ASR
performance, but no systematic evaluation of ASR diacritization performance,
independent of ASR performance, has been conducted to date. In this paper, we
attempt to experimentally clarify whether input diacritiztation indeed degrades
ASR quality, and to compare the diacritic recognition performance against
text-based diacritization as a post-processing step. We start with pre-trained
Arabic ASR models and fine-tune them on transcribed speech data with different
diacritization conditions: manual, automatic, and no diacritization. We isolate
diacritic recognition performance from the overall ASR performance using
coverage and precision metrics. We find that ASR diacritization significantly
outperforms text-based diacritization in post-processing, particularly when the
ASR model is fine-tuned with manually diacritized transcripts.
