  Retrieval-Augmented Generation (RAG), by incorporating external knowledge
with parametric memory of language models, has become the state-of-the-art
architecture for open-domain QA tasks. However, common knowledge bases are
inherently constrained by limited coverage and noisy information, making
retrieval-based approaches inadequate to answer implicit reasoning questions.
In this paper, we propose an Induction-Augmented Generation (IAG) framework
that utilizes inductive knowledge along with the retrieved documents for
implicit reasoning. We leverage large language models (LLMs) for deriving such
knowledge via a novel prompting method based on inductive reasoning patterns.
On top of this, we implement two versions of IAG named IAG-GPT and IAG-Student,
respectively. IAG-GPT directly utilizes the knowledge generated by GPT-3 for
answer prediction, while IAG-Student gets rid of dependencies on GPT service at
inference time by incorporating a student inductor model. The inductor is
firstly trained via knowledge distillation and further optimized by
back-propagating the generator feedback via differentiable beam scores.
Experimental results show that IAG outperforms RAG baselines as well as ChatGPT
on two Open-Domain QA tasks. Notably, our best models have won the first place
in the official leaderboards of CSQA2.0 (since Nov 1, 2022) and StrategyQA
(since Jan 8, 2023).
