  "Position bias" describes the tendency of users to interact with items on top
of a list with higher probability than with items at a lower position in the
list, regardless of the items' actual relevance. In the domain of recommender
systems, particularly recommender systems in digital libraries, position bias
has received little attention. We conduct a study in a real-world recommender
system that delivered ten million related-article recommendations to the users
of the digital library Sowiport, and the reference manager JabRef.
Recommendations were randomly chosen to be shuffled or non-shuffled, and we
compared click-through rate (CTR) for each rank of the recommendations.
According to our analysis, the CTR for the highest rank in the case of Sowiport
is 53% higher than expected in a hypothetical non-biased situation (0.189% vs.
0.123%). Similarly, in the case of Jabref the highest rank received a CTR of
1.276%, which is 87% higher than expected (0.683%). A chi-squared test confirms
the strong relationship between the rank of the recommendation shown to the
user and whether the user decided to click it (p < 0.01 for both Jabref and
Sowiport). Our study confirms the findings from other domains, that
recommendations in the top positions are more often clicked, regardless of
their actual relevance.
