  Cryptic crosswords, the dominant crossword variety in the UK, are a promising
target for advancing NLP systems that seek to process semantically complex,
highly compositional language. Cryptic clues read like fluent natural language
but are adversarially composed of two parts: a definition and a wordplay cipher
requiring character-level manipulations. Expert humans use creative
intelligence to solve cryptics, flexibly combining linguistic, world, and
domain knowledge. In this paper, we make two main contributions. First, we
present a dataset of cryptic clues as a challenging new benchmark for NLP
systems that seek to process compositional language in more creative,
human-like ways. After showing that three non-neural approaches and T5, a
state-of-the-art neural language model, do not achieve good performance, we
make our second main contribution: a novel curriculum approach, in which the
model is first fine-tuned on related tasks such as unscrambling words.We also
introduce a challenging data split, examine the meta-linguistic capabilities of
subword-tokenized models, and investigate model systematicity by perturbing the
wordplay part of clues, showing that T5 exhibits behavior partially consistent
with human solving strategies. Although our curricular approach considerably
improves on the T5 baseline, our best-performing model still fails to
generalize to the extent that humans can. Thus, cryptic crosswords remain an
unsolved challenge for NLP systems and a potential source of future innovation.
