  Large language models (LLMs) excel in many natural language processing (NLP)
tasks. However, since LLMs can only incorporate new knowledge through training
or supervised fine-tuning processes, they are unsuitable for applications that
demand precise, up-to-date, and private information not available in the
training corpora. This precise, up-to-date, and private information is
typically stored in relational databases. Thus, a promising solution is to
augment LLMs with the inclusion of relational databases as external memory.
This can ensure the timeliness, correctness, and consistency of data, and
assist LLMs in performing complex arithmetic operations beyond their inherent
capabilities. However, bridging the gap between LLMs and relational databases
is challenging. It requires the awareness of databases and data values stored
in databases to select correct databases and issue correct SQL queries.
Besides, it is necessary for the external memory to be independent of the LLM
to meet the needs of real-world applications. We introduce a novel LLM-agnostic
memory architecture comprising a database selection memory, a data value
memory, and relational databases. And we design an elegant pipeline to retrieve
information from it. Besides, we carefully design the prompts to instruct the
LLM to maximize the framework's potential. To evaluate our method, we compose a
new dataset with various types of questions. Experimental results show that our
framework enables LLMs to effectively answer database-related questions, which
is beyond their direct ability.
