  Language models suffer from various degenerate behaviors. These differ
between tasks: machine translation (MT) exhibits length bias, while tasks like
story generation exhibit excessive repetition. Recent work has attributed the
difference to task constrainedness, but evidence for this claim has always
involved many confounding variables. To study this question directly, we
introduce a new experimental framework that allows us to smoothly vary task
constrainedness, from MT at one end to fully open-ended generation at the
other, while keeping all other aspects fixed. We find that: (1) repetition
decreases smoothly with constrainedness, explaining the difference in
repetition across tasks; (2) length bias surprisingly also decreases with
constrainedness, suggesting some other cause for the difference in length bias;
(3) across the board, these problems affect the mode, not the whole
distribution; (4) the differences cannot be attributed to a change in the
entropy of the distribution, since another method of changing the entropy,
label smoothing, does not produce the same effect.
