  Reasoning about conjuncts in conjunctive sentences is important for a deeper
understanding of conjunctions in English and also how their usages and
semantics differ from conjunctive and disjunctive boolean logic. Existing NLI
stress tests do not consider non-boolean usages of conjunctions and use
templates for testing such model knowledge. Hence, we introduce ConjNLI, a
challenge stress-test for natural language inference over conjunctive
sentences, where the premise differs from the hypothesis by conjuncts removed,
added, or replaced. These sentences contain single and multiple instances of
coordinating conjunctions ("and", "or", "but", "nor") with quantifiers,
negations, and requiring diverse boolean and non-boolean inferences over
conjuncts. We find that large-scale pre-trained language models like RoBERTa do
not understand conjunctive semantics well and resort to shallow heuristics to
make inferences over such sentences. As some initial solutions, we first
present an iterative adversarial fine-tuning method that uses synthetically
created training data based on boolean and non-boolean heuristics. We also
propose a direct model advancement by making RoBERTa aware of predicate
semantic roles. While we observe some performance gains, ConjNLI is still
challenging for current methods, thus encouraging interesting future work for
better understanding of conjunctions. Our data and code are publicly available
at: https://github.com/swarnaHub/ConjNLI
