  Sequential Recommender Systems (SRSs) are widely employed to model user
behavior over time. However, their robustness in the face of perturbations in
training data remains a largely understudied yet critical issue. A fundamental
challenge emerges in previous studies aimed at assessing the robustness of
SRSs: the Rank-Biased Overlap (RBO) similarity is not particularly suited for
this task as it is designed for infinite rankings of items and thus shows
limitations in real-world scenarios. For instance, it fails to achieve a
perfect score of 1 for two identical finite-length rankings. To address this
challenge, we introduce a novel contribution: Finite Rank-Biased Overlap
(FRBO), an enhanced similarity tailored explicitly for finite rankings. This
innovation facilitates a more intuitive evaluation in practical settings. In
pursuit of our goal, we empirically investigate the impact of removing items at
different positions within a temporally ordered sequence. We evaluate two
distinct SRS models across multiple datasets, measuring their performance using
metrics such as Normalized Discounted Cumulative Gain (NDCG) and Rank List
Sensitivity. Our results demonstrate that removing items at the end of the
sequence has a statistically significant impact on performance, with NDCG
decreasing up to 60%. Conversely, removing items from the beginning or middle
has no significant effect. These findings underscore the criticality of the
position of perturbed items in the training data. As we spotlight the
vulnerabilities inherent in current SRSs, we fervently advocate for intensified
research efforts to fortify their robustness against adversarial perturbations.
