  Contemporary language models enable new opportunities for structured
reasoning with text, such as the construction and evaluation of intuitive,
proof-like textual entailment trees without relying on brittle formal logic.
However, progress in this direction has been hampered by a long-standing lack
of a clear protocol for determining what valid compositional entailment is.
This absence causes noisy datasets and limited performance gains by modern
neuro-symbolic engines. To address these problems, we formulate a consistent
and theoretically grounded approach to annotating decompositional entailment
datasets, and evaluate its impact on LLM-based textual inference. We find that
our resulting dataset, RDTE (Recognizing Decompositional Textual Entailment),
has a substantially higher internal consistency (+9%) than prior
decompositional entailment datasets, suggesting that RDTE is a significant step
forward in the long-standing problem of forming a clear protocol for discerning
entailment. We also find that training an RDTE-oriented entailment classifier
via knowledge distillation and employing it in a modern neuro-symbolic
reasoning engine significantly improves results (both accuracy and proof
quality) over other entailment classifier baselines, illustrating the practical
benefit of this advance for textual inference.
