  Streaming speech-to-avatar synthesis creates real-time animations for a
virtual character from audio data. Accurate avatar representations of speech
are important for the visualization of sound in linguistics, phonetics, and
phonology, visual feedback to assist second language acquisition, and virtual
embodiment for paralyzed patients. Previous works have highlighted the
capability of deep articulatory inversion to perform high-quality avatar
animation using electromagnetic articulography (EMA) features. However, these
models focus on offline avatar synthesis with recordings rather than real-time
audio, which is necessary for live avatar visualization or embodiment. To
address this issue, we propose a method using articulatory inversion for
streaming high quality facial and inner-mouth avatar animation from real-time
audio. Our approach achieves 130ms average streaming latency for every 0.1
seconds of audio with a 0.792 correlation with ground truth articulations.
Finally, we show generated mouth and tongue animations to demonstrate the
efficacy of our methodology.
