  Current language models are built on the so-called distributional semantic
approach to linguistic meaning that has the distributional hypothesis at its
core. The distributional hypothesis involves a holistic conception of word
meaning: the meaning of a word depends upon its relations to other words in the
model. A standard objection to meaning holism is the charge of instability: any
change in the meaning properties of a linguistic system (a human speaker, for
example) would lead to many changes or possibly a complete change in the entire
system. When the systems in question are trying to communicate with each other,
it has been argued that instability of this kind makes communication impossible
(Fodor and Lepore 1992, 1996, 1999). In this article, we examine whether the
instability objection poses a problem for distributional models of meaning.
First, we distinguish between distinct forms of instability that these models
could exhibit, and we argue that only one such form is relevant for
understanding the relation between instability and communication: what we call
differential instability. Differential instability is variation in the relative
distances between points in a space, rather than variation in the absolute
position of those points. We distinguish differential and absolute instability
by constructing two of our own models, a toy model constructed from the text of
two novels, and a more sophisticated model constructed using the Word2vec
algorithm from a combination of Wikipedia and SEP articles. We demonstrate the
two forms of instability by showing how these models change as the corpora they
are constructed from increase in size.
