  State of the art (SOTA) neural text to speech (TTS) models can generate
natural-sounding synthetic voices. These models are characterized by large
memory footprints and substantial number of operations due to the long-standing
focus on speech quality with cloud inference in mind. Neural TTS models are
generally not designed to perform standalone speech syntheses on
resource-constrained and no Internet access edge devices. In this work, an
efficient neural TTS called EfficientSpeech that synthesizes speech on an ARM
CPU in real-time is proposed. EfficientSpeech uses a shallow non-autoregressive
pyramid-structure transformer forming a U-Network. EfficientSpeech has 266k
parameters and consumes 90 MFLOPS only or about 1% of the size and amount of
computation in modern compact models such as Mixer-TTS. EfficientSpeech
achieves an average mel generation real-time factor of 104.3 on an RPi4. Human
evaluation shows only a slight degradation in audio quality as compared to
FastSpeech2.
