  Non-autoregressive (nAR) models for machine translation (MT) manifest
superior decoding speed when compared to autoregressive (AR) models, at the
expense of impaired fluency of their outputs. We improve the fluency of a nAR
model with connectionist temporal classification (CTC) by employing additional
features in the scoring model used during beam search decoding. Since the beam
search decoding in our model only requires to run the network in a single
forward pass, the decoding speed is still notably higher than in standard AR
models. We train models for three language pairs: German, Czech, and Romanian
from and into English. The results show that our proposed models can be more
efficient in terms of decoding speed and still achieve a competitive BLEU score
relative to AR models.
