  Language models (LM) are very powerful in lipreading systems. Language models
built upon the ground truth utterances of datasets learn grammar and structure
rules of words and sentences (the latter in the case of continuous speech).
However, visual co-articulation effects in visual speech signals damage the
performance of visual speech LM's as visually, people do not utter what the
language model expects. These models are commonplace but while higher-order
N-gram LM's may improve classification rates, the cost of this model is
disproportionate to the common goal of developing more accurate classifiers. So
we compare which unit would best optimize a lipreading (visual speech) LM to
observe their limitations. We compare three units; visemes (visual speech
units) \cite{lan2010improving}, phonemes (audible speech units), and words.
