  This paper introduces a novel combination of two tasks, previously treated
separately: acoustic-to-articulatory speech inversion (AAI) and
phoneme-to-articulatory (PTA) motion estimation. We refer to this joint task as
acoustic phoneme-to-articulatory speech inversion (APTAI) and explore two
different approaches, both working speaker- and text-independently during
inference. We use a multi-task learning setup, with the end-to-end goal of
taking raw speech as input and estimating the corresponding articulatory
movements, phoneme sequence, and phoneme alignment. While both proposed
approaches share these same requirements, they differ in their way of achieving
phoneme-related predictions: one is based on frame classification, the other on
a two-staged training procedure and forced alignment. We reach competitive
performance of 0.73 mean correlation for the AAI task and achieve up to
approximately 87% frame overlap compared to a state-of-the-art text-dependent
phoneme force aligner.
