  Sequential recommendation aims to model dynamic user behavior from historical
interactions. Existing methods rely on either explicit item IDs or general
textual features for sequence modeling to understand user preferences. While
promising, these approaches still struggle to model cold-start items or
transfer knowledge to new datasets. In this paper, we propose to model user
preferences and item features as language representations that can be
generalized to new items and datasets. To this end, we present a novel
framework, named Recformer, which effectively learns language representations
for sequential recommendation. Specifically, we propose to formulate an item as
a "sentence" (word sequence) by flattening item key-value attributes described
by text so that an item sequence for a user becomes a sequence of sentences.
For recommendation, Recformer is trained to understand the "sentence" sequence
and retrieve the next "sentence". To encode item sequences, we design a
bi-directional Transformer similar to the model Longformer but with different
embedding layers for sequential recommendation. For effective representation
learning, we propose novel pretraining and finetuning methods which combine
language understanding and recommendation tasks. Therefore, Recformer can
effectively recommend the next item based on language representations.
Extensive experiments conducted on six datasets demonstrate the effectiveness
of Recformer for sequential recommendation, especially in low-resource and
cold-start settings.
