  In the interest of interpreting neural NLI models and their reasoning
strategies, we carry out a systematic probing study which investigates whether
these models capture the crucial semantic features central to natural logic:
monotonicity and concept inclusion. Correctly identifying valid inferences in
downward-monotone contexts is a known stumbling block for NLI performance,
subsuming linguistic phenomena such as negation scope and generalized
quantifiers. To understand this difficulty, we emphasize monotonicity as a
property of a context and examine the extent to which models capture
monotonicity information in the contextual embeddings which are intermediate to
their decision making process. Drawing on the recent advancement of the probing
paradigm, we compare the presence of monotonicity features across various
models. We find that monotonicity information is notably weak in the
representations of popular NLI models which achieve high scores on benchmarks,
and observe that previous improvements to these models based on fine-tuning
strategies have introduced stronger monotonicity features together with their
improved performance on challenge sets.
