  The uniform information density (UID) hypothesis states that humans tend to
distribute information roughly evenly across an utterance or discourse. Early
evidence in support of the UID hypothesis came from Genzel & Charniak (2002),
which proposed an entropy rate constancy principle based on the probability of
English text under n-gram language models. We re-evaluate the claims of Genzel
& Charniak (2002) with neural language models, failing to find clear evidence
in support of entropy rate constancy. We conduct a range of experiments across
datasets, model sizes, and languages and discuss implications for the uniform
information density hypothesis and linguistic theories of efficient
communication more broadly.
