  Humans tend to follow the Uniform Information Density (UID) principle by
distributing information evenly in utterances. We study if decoding algorithms
implicitly follow this UID principle, and under what conditions adherence to
UID might be desirable for dialogue generation. We generate responses using
different decoding algorithms with GPT-2 on the Persona-Chat dataset and
collect human judgments on their quality using Amazon Mechanical Turk. We find
that (i) surprisingly, model-generated responses follow the UID principle to a
greater extent than human responses, and (ii) decoding algorithms that promote
UID do not generate higher-quality responses. Instead, when we control for
surprisal, non-uniformity of information density correlates with the quality of
responses with very low/high surprisal. Our findings indicate that encouraging
non-uniform responses is a potential solution to the ``likelihood trap''
problem (quality degradation in very high-likelihood text). Our dataset
containing multiple candidate responses per dialog history along with
human-annotated quality ratings is available at
https://huggingface.co/datasets/saranya132/dialog_uid_gpt2.
