  Language model is one of the most important modules in statistical machine
translation and currently the word-based language model dominants this
community. However, many translation models (e.g. phrase-based models) generate
the target language sentences by rendering and compositing the phrases rather
than the words. Thus, it is much more reasonable to model dependency between
phrases, but few research work succeed in solving this problem. In this paper,
we tackle this problem by designing a novel phrase-based language model which
attempts to solve three key sub-problems: 1, how to define a phrase in language
model; 2, how to determine the phrase boundary in the large-scale monolingual
data in order to enlarge the training set; 3, how to alleviate the data
sparsity problem due to the huge vocabulary size of phrases. By carefully
handling these issues, the extensive experiments on Chinese-to-English
translation show that our phrase-based language model can significantly improve
the translation quality by up to +1.47 absolute BLEU score.
