  Sequential recommendation (SR) has seen significant advancements with the
help of Pre-trained Language Models (PLMs). Some PLM-based SR models directly
use PLM to encode user historical behavior's text sequences to learn user
representations, while there is seldom an in-depth exploration of the
capability and suitability of PLM in behavior sequence modeling. In this work,
we first conduct extensive model analyses between PLMs and PLM-based SR models,
discovering great underutilization and parameter redundancy of PLMs in behavior
sequence modeling. Inspired by this, we explore different lightweight usages of
PLMs in SR, aiming to maximally stimulate the ability of PLMs for SR while
satisfying the efficiency and usability demands of practical systems. We
discover that adopting behavior-tuned PLMs for item initializations of
conventional ID-based SR models is the most economical framework of PLM-based
SR, which would not bring in any additional inference cost but could achieve a
dramatic performance boost compared with the original version. Extensive
experiments on five datasets show that our simple and universal framework leads
to significant improvement compared to classical SR and SOTA PLM-based SR
models without additional inference costs.
