  We present an extensive empirical comparison of several smoothing techniques
in the domain of language modeling, including those described by Jelinek and
Mercer (1980), Katz (1987), and Church and Gale (1991). We investigate for the
first time how factors such as training data size, corpus (e.g., Brown versus
Wall Street Journal), and n-gram order (bigram versus trigram) affect the
relative performance of these methods, which we measure through the
cross-entropy of test data. In addition, we introduce two novel smoothing
techniques, one a variation of Jelinek-Mercer smoothing and one a very simple
linear interpolation technique, both of which outperform existing methods.
