  In this short note we explore what is needed for the unsupervised training of
graph language models based on link grammars. First, we introduce the
ter-mination tags formalism required to build a language model based on a link
grammar formalism of Sleator and Temperley [21] and discuss the influence of
context on the unsupervised learning of link grammars. Second, we pro-pose a
statistical link grammar formalism, allowing for statistical language
generation. Third, based on the above formalism, we show that the classical
dissertation of Yuret [25] on discovery of linguistic relations using lexical
at-traction ignores contextual properties of the language, and thus the
approach to unsupervised language learning relying just on bigrams is flawed.
This correlates well with the unimpressive results in unsupervised training of
graph language models based on bigram approach of Yuret.
