  Fast inference speed is an important goal towards real-world deployment of
speech translation (ST) systems. End-to-end (E2E) models based on the
encoder-decoder architecture are more suitable for this goal than traditional
cascaded systems, but their effectiveness regarding decoding speed has not been
explored so far. Inspired by recent progress in non-autoregressive (NAR)
methods in text-based translation, which generates target tokens in parallel by
eliminating conditional dependencies, we study the problem of NAR decoding for
E2E-ST. We propose a novel NAR E2E-ST framework, Orthros, in which both NAR and
autoregressive (AR) decoders are jointly trained on the shared speech encoder.
The latter is used for selecting better translation among various length
candidates generated from the former, which dramatically improves the
effectiveness of a large length beam with negligible overhead. We further
investigate effective length prediction methods from speech inputs and the
impact of vocabulary sizes. Experiments on four benchmarks show the
effectiveness of the proposed method in improving inference speed while
maintaining competitive translation quality compared to state-of-the-art AR
E2E-ST systems.
