  In this paper, we study a multi-step interactive recommendation problem,
where the item recommended at current step may affect the quality of future
recommendations. To address the problem, we develop a novel and effective
approach, named CFRL, which seamlessly integrates the ideas of both
collaborative filtering (CF) and reinforcement learning (RL). More
specifically, we first model the recommender-user interactive recommendation
problem as an agent-environment RL task, which is mathematically described by a
Markov decision process (MDP). Further, to achieve collaborative
recommendations for the entire user community, we propose a novel CF-based MDP
by encoding the states of all users into a shared latent vector space. Finally,
we propose an effective Q-network learning method to learn the agent's optimal
policy based on the CF-based MDP. The capability of CFRL is demonstrated by
comparing its performance against a variety of existing methods on real-world
datasets.
