  Sequential recommendation is an important recommendation task that aims to
predict the next item in a sequence. Recently, adaptations of language models,
particularly Transformer-based models such as SASRec and BERT4Rec, have
achieved state-of-the-art results in sequential recommendation. In these
models, item ids replace tokens in the original language models. However, this
approach has limitations. First, the vocabulary of item ids may be many times
larger than in language models. Second, the classical Top-K recommendation
approach used by these models may not be optimal for complex recommendation
objectives, including auxiliary objectives such as diversity, coverage or
coherence. Recent progress in generative language models inspires us to revisit
generative approaches to address these challenges. This paper presents the
GPTRec sequential recommendation model, which is based on the GPT-2
architecture. GPTRec can address large vocabulary issues by splitting item ids
into sub-id tokens using a novel SVD Tokenisation algorithm based on quantised
item embeddings from an SVD decomposition of the user-item interaction matrix.
The paper also presents a novel Next-K recommendation strategy, which generates
recommendations item-by-item, considering already recommended items. The Next-K
strategy can be used for producing complex interdependent recommendation lists.
We experiment with GPTRec on the MovieLens-1M dataset and show that using
sub-item tokenisation GPTRec can match the quality of SASRec while reducing the
embedding table by 40%. We also show that the recommendations generated by
GPTRec on MovieLens-1M using the Next-K recommendation strategy match the
quality of SASRec in terms of NDCG@10, meaning that the model can serve as a
strong starting point for future research.
