  Language-enabled AI systems can answer complex, multi-hop questions to high
accuracy, but supporting answers with evidence is a more challenging task which
is important for the transparency and trustworthiness to users. Prior work in
this area typically makes a trade-off between efficiency and accuracy;
state-of-the-art deep neural network systems are too cumbersome to be useful in
large-scale applications, while the fastest systems lack reliability. In this
work, we integrate fast syntactic methods with powerful semantic methods for
multi-hop explanation generation based on declarative facts. Our best system,
which learns a lightweight operation to simulate multi-hop reasoning over
pieces of evidence and fine-tunes language models to re-rank generated
explanation chains, outperforms a purely syntactic baseline from prior work by
up to 7% in gold explanation retrieval rate.
