  Item representation learning (IRL) plays an essential role in recommender
systems, especially for sequential recommendation. Traditional sequential
recommendation models usually utilize ID embeddings to represent items, which
are not shared across different domains and lack the transferable ability.
Recent studies use pre-trained language models (PLM) for item text embeddings
(text-based IRL) that are universally applicable across domains. However, the
existing text-based IRL is unaware of the important collaborative filtering
(CF) information. In this paper, we propose CoWPiRec, an approach of
Collaborative Word-based Pre-trained item representation for Recommendation. To
effectively incorporate CF information into text-based IRL, we convert the
item-level interaction data to a word graph containing word-level
collaborations. Subsequently, we design a novel pre-training task to align the
word-level semantic- and CF-related item representation. Extensive experimental
results on multiple public datasets demonstrate that compared to
state-of-the-art transferable sequential recommenders, CoWPiRec achieves
significantly better performances in both fine-tuning and zero-shot settings
for cross-scenario recommendation and effectively alleviates the cold-start
issue. The code is available at: https://github.com/ysh-1998/CoWPiRec.
