  Reinforcement learning based recommender systems (RL-based RS) aim at
learning a good policy from a batch of collected data, by casting
recommendations to multi-step decision-making tasks. However, current RL-based
RS research commonly has a large reality gap. In this paper, we introduce the
first open-source real-world dataset, RL4RS, hoping to replace the artificial
datasets and semi-simulated RS datasets previous studies used due to the
resource limitation of the RL-based RS domain. Unlike academic RL research,
RL-based RS suffers from the difficulties of being well-validated before
deployment. We attempt to propose a new systematic evaluation framework,
including evaluation of environment simulation, evaluation on environments,
counterfactual policy evaluation, and evaluation on environments built from
test set. In summary, the RL4RS (Reinforcement Learning for Recommender
Systems), a new resource with special concerns on the reality gaps, contains
two real-world datasets, data understanding tools, tuned simulation
environments, related advanced RL baselines, batch RL baselines, and
counterfactual policy evaluation algorithms. The RL4RS suite can be found at
https://github.com/fuxiAIlab/RL4RS. In addition to the RL-based recommender
systems, we expect the resource to contribute to research in applied
reinforcement learning.
