  We present a fully on-device streaming Speech2Speech conversion model that
normalizes a given input speech directly to synthesized output speech.
Deploying such a model on mobile devices pose significant challenges in terms
of memory footprint and computation requirements. We present a streaming-based
approach to produce an acceptable delay, with minimal loss in speech conversion
quality, when compared to a reference state of the art non-streaming approach.
Our method consists of first streaming the encoder in real time while the
speaker is speaking. Then, as soon as the speaker stops speaking, we run the
spectrogram decoder in streaming mode along the side of a streaming vocoder to
generate output speech. To achieve an acceptable delay-quality trade-off, we
propose a novel hybrid approach for look-ahead in the encoder which combines a
look-ahead feature stacker with a look-ahead self-attention. We show that our
streaming approach is almost 2x faster than real time on the Pixel4 CPU.
