  Simultaneous speech-to-speech translation is widely useful but extremely
challenging, since it needs to generate target-language speech concurrently
with the source-language speech, with only a few seconds delay. In addition, it
needs to continuously translate a stream of sentences, but all recent solutions
merely focus on the single-sentence scenario. As a result, current approaches
accumulate latencies progressively when the speaker talks faster, and introduce
unnatural pauses when the speaker talks slower. To overcome these issues, we
propose Self-Adaptive Translation (SAT) which flexibly adjusts the length of
translations to accommodate different source speech rates. At similar levels of
translation quality (as measured by BLEU), our method generates more fluent
target speech (as measured by the naturalness metric MOS) with substantially
lower latency than the baseline, in both Zh <-> En directions.
