  The multinomial language model has been one of the most effective models of
retrieval for over a decade. However, the multinomial distribution does not
model one important linguistic phenomenon relating to term-dependency, that is
the tendency of a term to repeat itself within a document (i.e. word
burstiness). In this article, we model document generation as a random process
with reinforcement (a multivariate Polya process) and develop a Dirichlet
compound multinomial language model that captures word burstiness directly.
  We show that the new reinforced language model can be computed as efficiently
as current retrieval models, and with experiments on an extensive set of TREC
collections, we show that it significantly outperforms the state-of-the-art
language model for a number of standard effectiveness metrics. Experiments also
show that the tuning parameter in the proposed model is more robust than in the
multinomial language model. Furthermore, we develop a constraint for the
verbosity hypothesis and show that the proposed model adheres to the
constraint. Finally, we show that the new language model essentially introduces
a measure closely related to idf which gives theoretical justification for
combining the term and document event spaces in tf-idf type schemes.
