  Undirected neural sequence models have achieved performance competitive with
the state-of-the-art directed sequence models that generate monotonically from
left to right in machine translation tasks. In this work, we train a policy
that learns the generation order for a pre-trained, undirected translation
model via reinforcement learning. We show that the translations decoded by our
learned orders achieve higher BLEU scores than the outputs decoded from left to
right or decoded by the learned order from Mansimov et al. (2019) on the WMT'14
German-English translation task. On examples with a maximum source and target
length of 30 from De-En, WMT'16 English-Romanian, and WMT'21 English-Chinese
translation tasks, our learned order outperforms all heuristic generation
orders on four out of six tasks. We next carefully analyze the learned order
patterns via qualitative and quantitative analysis. We show that our policy
generally follows an outer-to-inner order, predicting the left-most and
right-most positions first, and then moving toward the middle while skipping
less important words at the beginning. Furthermore, the policy usually predicts
positions for a single syntactic constituent structure in consecutive steps. We
believe our findings could provide more insights on the mechanism of undirected
generation models and encourage further research in this direction. Our code is
publicly available at https://github.com/jiangycTarheel/undirected-generation
