  We study the possibilities of building a non-autoregressive speech-to-text
translation model using connectionist temporal classification (CTC), and use
CTC-based automatic speech recognition as an auxiliary task to improve the
performance. CTC's success on translation is counter-intuitive due to its
monotonicity assumption, so we analyze its reordering capability. Kendall's tau
distance is introduced as the quantitative metric, and gradient-based
visualization provides an intuitive way to take a closer look into the model.
Our analysis shows that transformer encoders have the ability to change the
word order and points out the future research direction that worth being
explored more on non-autoregressive speech translation.
