  Retrieval-Augmented Generation (RAG) demonstrates great value in alleviating
outdated knowledge or hallucination by supplying LLMs with updated and relevant
knowledge. However, there are still several difficulties for RAG in
understanding complex multi-hop query and retrieving relevant documents, which
require LLMs to perform reasoning and retrieve step by step. Inspired by
human's reasoning process in which they gradually search for the required
information, it is natural to ask whether the LLMs could notice the missing
information in each reasoning step. In this work, we first experimentally
verified the ability of LLMs to extract information as well as to know the
missing. Based on the above discovery, we propose a Missing Information Guided
Retrieve-Extraction-Solving paradigm (MIGRES), where we leverage the
identification of missing information to generate a targeted query that steers
the subsequent knowledge retrieval. Besides, we design a sentence-level
re-ranking filtering approach to filter the irrelevant content out from
document, along with the information extraction capability of LLMs to extract
useful information from cleaned-up documents, which in turn to bolster the
overall efficacy of RAG. Extensive experiments conducted on multiple public
datasets reveal the superiority of the proposed MIGRES method, and analytical
experiments demonstrate the effectiveness of our proposed modules.
