  We investigate the automatic processing of child speech therapy sessions
using ultrasound visual biofeedback, with a specific focus on complementing
acoustic features with ultrasound images of the tongue for the tasks of speaker
diarization and time-alignment of target words. For speaker diarization, we
propose an ultrasound-based time-domain signal which we call estimated tongue
activity. For word-alignment, we augment an acoustic model with low-dimensional
representations of ultrasound images of the tongue, learned by a convolutional
neural network. We conduct our experiments using the Ultrasuite repository of
ultrasound and speech recordings for child speech therapy sessions. For both
tasks, we observe that systems augmented with ultrasound data outperform
corresponding systems using only the audio signal.
