  Large language models (LLMs) have demonstrated great success in various
fields, benefiting from their huge amount of parameters that store knowledge.
However, LLMs still suffer from several key issues, such as hallucination
problems, knowledge update issues, and lacking domain-specific expertise. The
appearance of retrieval-augmented generation (RAG), which leverages an external
knowledge database to augment LLMs, makes up those drawbacks of LLMs. This
paper reviews all significant techniques of RAG, especially in the retriever
and the retrieval fusions. Besides, tutorial codes are provided for
implementing the representative techniques in RAG. This paper further discusses
the RAG training, including RAG with/without datastore update. Then, we
introduce the application of RAG in representative natural language processing
tasks and industrial scenarios. Finally, this paper discusses the future
directions and challenges of RAG for promoting its development.
