  Despite the success of integrating large language models into the development
of conversational systems, many studies have shown the effectiveness of
retrieving and augmenting external knowledge for informative responses. Hence,
many existing studies commonly assume the always need for Retrieval Augmented
Generation (RAG) in a conversational system without explicit control. This
raises a research question about such a necessity. In this study, we propose to
investigate the need for each turn of system response to be augmented with
external knowledge. In particular, by leveraging human judgements on the binary
choice of adaptive augmentation, we develop RAGate, a gating model, which
models conversation context and relevant inputs to predict if a conversational
system requires RAG for improved responses. We conduct extensive experiments on
devising and applying RAGate to conversational models and well-rounded analyses
of different conversational scenarios. Our experimental results and analysis
indicate the effective application of RAGate in RAG-based conversational
systems in identifying system responses for appropriate RAG with high-quality
responses and a high generation confidence. This study also identifies the
correlation between the generation's confidence level and the relevance of the
augmented knowledge.
