  Standard evaluations of deep learning models for semantics using naturalistic
corpora are limited in what they can tell us about the fidelity of the learned
representations, because the corpora rarely come with good measures of semantic
complexity. To overcome this limitation, we present a method for generating
data sets of multiply-quantified natural language inference (NLI) examples in
which semantic complexity can be precisely characterized, and we use this
method to show that a variety of common architectures for NLI inevitably fail
to encode crucial information; only a model with forced lexical alignments
avoids this damaging information loss.
