  Most sequence-to-sequence (seq2seq) models are autoregressive; they generate
each token by conditioning on previously generated tokens. In contrast,
non-autoregressive seq2seq models generate all tokens in one pass, which leads
to increased efficiency through parallel processing on hardware such as GPUs.
However, directly modeling the joint distribution of all tokens simultaneously
is challenging, and even with increasingly complex model structures accuracy
lags significantly behind autoregressive models. In this paper, we propose a
simple, efficient, and effective model for non-autoregressive sequence
generation using latent variable models. Specifically, we turn to generative
flow, an elegant technique to model complex distributions using neural
networks, and design several layers of flow tailored for modeling the
conditional density of sequential latent variables. We evaluate this model on
three neural machine translation (NMT) benchmark datasets, achieving comparable
performance with state-of-the-art non-autoregressive NMT models and almost
constant decoding time w.r.t the sequence length.
