  In recent years there has been a burgeoning interest in the use of
computational methods to distinguish between elicited speech samples produced
by patients with dementia, and those from healthy controls. The difference
between perplexity estimates from two neural language models (LMs) - one
trained on transcripts of speech produced by healthy participants and the other
trained on transcripts from patients with dementia - as a single feature for
diagnostic classification of unseen transcripts has been shown to produce
state-of-the-art performance. However, little is known about why this approach
is effective, and on account of the lack of case/control matching in the most
widely-used evaluation set of transcripts (DementiaBank), it is unclear if
these approaches are truly diagnostic, or are sensitive to other variables. In
this paper, we interrogate neural LMs trained on participants with and without
dementia using synthetic narratives previously developed to simulate
progressive semantic dementia by manipulating lexical frequency. We find that
perplexity of neural LMs is strongly and differentially associated with lexical
frequency, and that a mixture model resulting from interpolating control and
dementia LMs improves upon the current state-of-the-art for models trained on
transcript text exclusively.
