  Observations on the past provide some hints about what will happen in the
future, and this can be quantified using information theory. The ``predictive
information'' defined in this way has connections to measures of complexity
that have been proposed both in the study of dynamical systems and in
mathematical statistics. In particular, the predictive information diverges
when the observed data stream allows us to learn an increasingly precise model
for the dynamics that generate the data, and the structure of this divergence
measures the complexity of the model. We argue that divergent contributions to
the predictive information provide the only measure of complexity or richness
that is consistent with certain plausible requirements.
