  The integration of large language models (LLMs) and search engines represents
a significant evolution in knowledge acquisition methodologies. However,
determining the knowledge that an LLM already possesses and the knowledge that
requires the help of a search engine remains an unresolved issue. Most existing
methods solve this problem through the results of preliminary answers or
reasoning done by the LLM itself, but this incurs excessively high
computational costs. This paper introduces a novel collaborative approach,
namely SlimPLM, that detects missing knowledge in LLMs with a slim proxy model,
to enhance the LLM's knowledge acquisition process. We employ a proxy model
which has far fewer parameters, and take its answers as heuristic answers.
Heuristic answers are then utilized to predict the knowledge required to answer
the user question, as well as the known and unknown knowledge within the LLM.
We only conduct retrieval for the missing knowledge in questions that the LLM
does not know. Extensive experimental results on five datasets with two LLMs
demonstrate a notable improvement in the end-to-end performance of LLMs in
question-answering tasks, achieving or surpassing current state-of-the-art
models with lower LLM inference costs.
