  Non-autoregressive neural machine translation (NAT) generates each target
word in parallel and has achieved promising inference acceleration. However,
existing NAT models still have a big gap in translation quality compared to
autoregressive neural machine translation models due to the enormous decoding
space. To address this problem, we propose a novel NAT framework named
ReorderNAT which explicitly models the reordering information in the decoding
procedure. We further introduce deterministic and non-deterministic decoding
strategies that utilize reordering information to narrow the decoding search
space in our proposed ReorderNAT. Experimental results on various widely-used
datasets show that our proposed model achieves better performance compared to
existing NAT models, and even achieves comparable translation quality as
autoregressive translation models with a significant speedup.
