  Personalization, the ability to tailor a system to individual users, is an
essential factor in user experience with natural language processing (NLP)
systems. With the emergence of Large Language Models (LLMs), a key question is
how to leverage these models to better personalize user experiences. To
personalize a language model's output, a straightforward approach is to
incorporate past user data into the language model prompt, but this approach
can result in lengthy inputs exceeding limitations on input length and
incurring latency and cost issues. Existing approaches tackle such challenges
by selectively extracting relevant user data (i.e. selective retrieval) to
construct a prompt for downstream tasks. However, retrieval-based methods are
limited by potential information loss, lack of more profound user
understanding, and cold-start challenges. To overcome these limitations, we
propose a novel summary-augmented approach by extending retrieval-augmented
personalization with task-aware user summaries generated by LLMs. The summaries
can be generated and stored offline, enabling real-world systems with runtime
constraints like voice assistants to leverage the power of LLMs. Experiments
show our method with 75% less of retrieved user data is on-par or outperforms
retrieval augmentation on most tasks in the LaMP personalization benchmark. We
demonstrate that offline summarization via LLMs and runtime retrieval enables
better performance for personalization on a range of tasks under practical
constraints.
