  Compositionality in language models presents a problem when processing
idiomatic expressions, as their meaning often cannot be directly derived from
their individual parts. Although fine-tuning and other optimization strategies
can be used to improve representations of idiomatic expressions, this depends
on the availability of relevant data. We present the Noun Compound Synonym
Substitution in Books - NCSSB - datasets, which are created by substitution of
synonyms of potentially idiomatic English noun compounds in public domain book
texts. We explore the trade-off between data quantity and quality when training
models for idiomaticity detection, in conjunction with contextual information
obtained locally (from the surrounding sentences) or externally (through
language resources). Performance on an idiomaticity detection task indicates
that dataset quality is a stronger factor for context-enriched models, but that
quantity also plays a role in models without context inclusion strategies.
