  In academic research, recommender systems are often evaluated on benchmark
datasets, without much consideration about the global timeline. Hence, we are
unable to answer questions like: Do loyal users enjoy better recommendations
than non-loyal users? Loyalty can be defined by the time period a user has been
active in a recommender system, or by the number of historical interactions a
user has. In this paper, we offer a comprehensive analysis of recommendation
results along global timeline. We conduct experiments with five widely used
models, i.e., BPR, NeuMF, LightGCN, SASRec and TiSASRec, on four benchmark
datasets, i.e., MovieLens-25M, Yelp, Amazon-music, and Amazon-electronic. Our
experiment results give an answer "No" to the above question. Users with many
historical interactions suffer from relatively poorer recommendations. Users
who stay with the system for a shorter time period enjoy better
recommendations. Both findings are counter-intuitive. Interestingly, users who
have recently interacted with the system, with respect to the time point of the
test instance, enjoy better recommendations. The finding on recency applies to
all users, regardless of users' loyalty. Our study offers a different
perspective to understand recommender accuracy, and our findings could trigger
a revisit of recommender model design. The code is available in
\url{https://github.com/putatu/recommenderLoyalty.
