  Making the content generated by Large Language Model (LLM), accurate,
credible and traceable is crucial, especially in complex knowledge-intensive
tasks that require multi-step reasoning and each step needs knowledge to solve.
Retrieval-augmented generation is good potential to solve this problem.
However, where and how to introduce Information Retrieval (IR) to LLM is a big
challenge. Previous work has the problems that wrong knowledge retrieved by IR
misleads the LLM and interaction between IR and LLM breaks the reasoning chain
of LLM. This paper proposes a novel framework named
\textbf{Search-in-the-Chain} (SearChain) for the interaction between LLM and IR
to solve the challenges. First, LLM generates the reasoning chain named
Chain-of-Query (CoQ) where each node consists of an IR-oriented query-answer
pair. Second, IR verifies the answer of each node of CoQ. It corrects the
answer that is not consistent with the retrieved information when IR gives high
confidence, which improves the credibility. Third, LLM can indicate its missing
knowledge in CoQ and rely on IR to provide this knowledge to LLM. These
operations improve the accuracy in terms of reasoning and knowledge. Finally,
SearChain generates the reasoning process and marks references to supporting
documents for each reasoning step, which improves traceability. Interaction
with IR in SearChain forms a novel reasoning path based on a tree, which
enables LLM to dynamically modify the direction of reasoning. Experiments show
that SearChain outperforms state-of-the-art baselines on complex
knowledge-intensive tasks including multi-hop Q\&A, slot filling, fact
checking, and long-form Q\&A.
