  Recent work on word ordering has argued that syntactic structure is
important, or even required, for effectively recovering the order of a
sentence. We find that, in fact, an n-gram language model with a simple
heuristic gives strong results on this task. Furthermore, we show that a long
short-term memory (LSTM) language model is even more effective at recovering
order, with our basic model outperforming a state-of-the-art syntactic model by
11.5 BLEU points. Additional data and larger beams yield further gains, at the
expense of training and search time.
