  Five years after the first published proofs of concept, direct approaches to
speech translation (ST) are now competing with traditional cascade solutions.
In light of this steady progress, can we claim that the performance gap between
the two is closed? Starting from this question, we present a systematic
comparison between state-of-the-art systems representative of the two
paradigms. Focusing on three language directions
(English-German/Italian/Spanish), we conduct automatic and manual evaluations,
exploiting high-quality professional post-edits and annotations. Our
multi-faceted analysis on one of the few publicly available ST benchmarks
attests for the first time that: i) the gap between the two paradigms is now
closed, and ii) the subtle differences observed in their behavior are not
sufficient for humans neither to distinguish them nor to prefer one over the
other.
