  Recommender systems (RSs) have become an inseparable part of our everyday
lives. They help us find our favorite items to purchase, our friends on social
networks, and our favorite movies to watch. Traditionally, the recommendation
problem was considered to be a classification or prediction problem, but it is
now widely agreed that formulating it as a sequential decision problem can
better reflect the user-system interaction. Therefore, it can be formulated as
a Markov decision process (MDP) and be solved by reinforcement learning (RL)
algorithms. Unlike traditional recommendation methods, including collaborative
filtering and content-based filtering, RL is able to handle the sequential,
dynamic user-system interaction and to take into account the long-term user
engagement. Although the idea of using RL for recommendation is not new and has
been around for about two decades, it was not very practical, mainly because of
scalability problems of traditional RL algorithms. However, a new trend has
emerged in the field since the introduction of deep reinforcement learning
(DRL), which made it possible to apply RL to the recommendation problem with
large state and action spaces. In this paper, a survey on reinforcement
learning based recommender systems (RLRSs) is presented. Our aim is to present
an outlook on the field and to provide the reader with a fairly complete
knowledge of key concepts of the field. We first recognize and illustrate that
RLRSs can be generally classified into RL- and DRL-based methods. Then, we
propose an RLRS framework with four components, i.e., state representation,
policy optimization, reward formulation, and environment building, and survey
RLRS algorithms accordingly. We highlight emerging topics and depict important
trends using various graphs and tables. Finally, we discuss important aspects
and challenges that can be addressed in the future.
