  The long-tail recommendation is a challenging task for traditional
recommender systems, due to data sparsity and data imbalance issues. The recent
development of large language models (LLMs) has shown their abilities in
complex reasoning, which can help to deduce users' preferences based on very
few previous interactions. However, since most LLM-based systems rely on items'
semantic meaning as the sole evidence for reasoning, the collaborative
information of user-item interactions is neglected, which can cause the LLM's
reasoning to be misaligned with task-specific collaborative information of the
dataset. To further align LLMs' reasoning to task-specific user-item
interaction knowledge, we introduce collaborative retrieval-augmented LLMs,
CoRAL, which directly incorporate collaborative evidence into the prompts.
Based on the retrieved user-item interactions, the LLM can analyze shared and
distinct preferences among users, and summarize the patterns indicating which
types of users would be attracted by certain items. The retrieved collaborative
evidence prompts the LLM to align its reasoning with the user-item interaction
patterns in the dataset. However, since the capacity of the input prompt is
limited, finding the minimally-sufficient collaborative information for
recommendation tasks can be challenging. We propose to find the optimal
interaction set through a sequential decision-making process and develop a
retrieval policy learned through a reinforcement learning (RL) framework,
CoRAL. Our experimental results show that CoRAL can significantly improve LLMs'
reasoning abilities on specific recommendation tasks. Our analysis also reveals
that CoRAL can more efficiently explore collaborative information through
reinforcement learning.
