  Sensitising language models (LMs) to external context helps them to more
effectively capture the speaking patterns of individuals with specific
characteristics or in particular environments. This work investigates to what
extent rich character and film annotations can be leveraged to personalise LMs
in a scalable manner. We then explore the use of such models in evaluating
context specificity in machine translation. We build LMs which leverage rich
contextual information to reduce perplexity by up to 6.5% compared to a
non-contextual model, and generalise well to a scenario with no
speaker-specific data, relying on combinations of demographic characteristics
expressed via metadata. Our findings are consistent across two corpora, one of
which (Cornell-rich) is also a contribution of this paper. We then use our
personalised LMs to measure the co-occurrence of extra-textual context and
translation hypotheses in a machine translation setting. Our results suggest
that the degree to which professional translations in our domain are
context-specific can be preserved to a better extent by a contextual machine
translation model than a non-contextual model, which is also reflected in the
contextual model's superior reference-based scores.
