  Natural language inference has trended toward studying contexts beyond the
sentence level. An important application area is law: past cases often do not
foretell how they apply to new situations and implications must be inferred.
This paper introduces LawngNLI, constructed from U.S. legal opinions with
automatic labels with high human-validated accuracy. Premises are long and
multigranular. Experiments show two use cases. First, LawngNLI can benchmark
for in-domain generalization from short to long contexts. It has remained
unclear if large-scale long-premise NLI datasets actually need to be
constructed: near-top performance on long premises could be achievable by
fine-tuning using short premises. Without multigranularity, benchmarks cannot
distinguish lack of fine-tuning on long premises versus domain shift between
short and long datasets. In contrast, our long and short premises share the
same examples and domain. Models fine-tuned using several past NLI datasets
and/or our short premises fall short of top performance on our long premises.
So for at least certain domains (such as ours), large-scale long-premise
datasets are needed. Second, LawngNLI can benchmark for implication-based
retrieval. Queries are entailed or contradicted by target documents, allowing
users to move between arguments and evidence. Leading retrieval models perform
reasonably zero shot on a LawngNLI-derived retrieval task. We compare different
systems for re-ranking, including lexical overlap and cross-encoders fine-tuned
using a modified LawngNLI or past NLI datasets. LawngNLI can train and test
systems for implication-based case retrieval and argumentation.
