  The distribution of sentence length in ordinary language is not well captured
by the existing models. Here we survey previous models of sentence length and
present our random walk model that offers both a better fit with the data and a
better understanding of the distribution. We develop a generalization of KL
divergence, discuss measuring the noise inherent in a corpus, and present a
hyperparameter-free Bayesian model comparison method that has strong conceptual
ties to Minimal Description Length modeling. The models we obtain require only
a few dozen bits, orders of magnitude less than the naive nonparametric MDL
models would.
