  The meaning of polysemous words often varies in a highly productive yet
predictable way. Generalizing the regularity between conventional senses to
derive novel word meaning is crucial for automated processing of non-literal
language uses such as figurative expressions. We introduce a novel task called
systematic word meta-sense extension (SWORME) to test and improve language
models' ability to extend word meaning to denote new semantic domains (also
called meta-senses) that bear regular semantic relations with existing senses.
We found that language models prefer incremental lexical semantic change toward
conceptually similar meta-senses such as logical metonymy, and are much worse
at predicting highly non-literal meaning extensions such as metaphors. We
propose a novel analogy-based method of word meaning extension, and show that
it effectively improves language model systematicity in making both gradual and
radical types of meta-sense extension. We further demonstrate that learning
systematic meta-sense extensions benefits language models on multiple
benchmarks of figurative language understanding.
