  Text-based collaborative filtering (TCF) has become the mainstream approach
for text and news recommendation, utilizing text encoders, also known as
language models (LMs), to represent items. However, existing TCF models
primarily focus on using small or medium-sized LMs. It remains uncertain what
impact replacing the item encoder with one of the largest and most powerful
LMs, such as the 175-billion parameter GPT-3 model, would have on
recommendation performance. Can we expect unprecedented results? To this end,
we conduct an extensive series of experiments aimed at exploring the
performance limits of the TCF paradigm. Specifically, we increase the size of
item encoders from one hundred million to one hundred billion to reveal the
scaling limits of the TCF paradigm. We then examine whether these extremely
large LMs could enable a universal item representation for the recommendation
task. Furthermore, we compare the performance of the TCF paradigm utilizing the
most powerful LMs to the currently dominant ID embedding-based paradigm and
investigate the transferability of this TCF paradigm. Finally, we compare TCF
with the recently popularized prompt-based recommendation using ChatGPT. Our
research findings have not only yielded positive results but also uncovered
some surprising and previously unknown negative outcomes, which can inspire
deeper reflection and innovative thinking regarding text-based recommender
systems. Codes and datasets will be released for further research.
