  Neural text generation, including neural machine translation, image
captioning, and summarization, has been quite successful recently. However,
during training time, typically only one reference is considered for each
example, even though there are often multiple references available, e.g., 4
references in NIST MT evaluations, and 5 references in image captioning data.
We first investigate several different ways of utilizing multiple human
references during training. But more importantly, we then propose an algorithm
to generate exponentially many pseudo-references by first compressing existing
human references into lattices and then traversing them to generate new
pseudo-references. These approaches lead to substantial improvements over
strong baselines in both machine translation (+1.5 BLEU) and image captioning
(+3.1 BLEU / +11.7 CIDEr).
