  Modeling long histories plays a pivotal role in enhancing recommendation
systems, allowing to capture user's evolving preferences, resulting in more
precise and personalized recommendations. In this study we tackle the
challenges of modeling long user histories for preference understanding in
natural language. Specifically, we introduce a new User Embedding Module (UEM)
that efficiently processes user history in free-form text by compressing and
representing them as embeddings, to use them as soft prompts to a LM. Our
experiments demonstrate the superior capability of this approach in handling
significantly longer histories compared to conventional text based prompting
methods, yielding substantial improvements in predictive performance. The main
contribution of this research is to demonstrate the ability to bias language
models with user signals represented as embeddings.
