  Recommender Systems are nowadays successfully used by all major web sites
(from e-commerce to social media) to filter content and make suggestions in a
personalized way. Academic research largely focuses on the value of
recommenders for consumers, e.g., in terms of reduced information overload. To
what extent and in which ways recommender systems create business value is,
however, much less clear, and the literature on the topic is scattered. In this
research commentary, we review existing publications on field tests of
recommender systems and report which business-related performance measures were
used in such real-world deployments. We summarize common challenges of
measuring the business value in practice and critically discuss the value of
algorithmic improvements and offline experiments as commonly done in academic
environments. Overall, our review indicates that various open questions remain
both regarding the realistic quantification of the business effects of
recommenders and the performance assessment of recommendation algorithms in
academia.
