  Reinforcement learning (RL) has proven its worth in a series of artificial
domains, and is beginning to show some successes in real-world scenarios.
However, much of the research advances in RL are often hard to leverage in
real-world systems due to a series of assumptions that are rarely satisfied in
practice. We present a set of nine unique challenges that must be addressed to
productionize RL to real world problems. For each of these challenges, we
specify the exact meaning of the challenge, present some approaches from the
literature, and specify some metrics for evaluating that challenge. An approach
that addresses all nine challenges would be applicable to a large number of
real world problems. We also present an example domain that has been modified
to present these challenges as a testbed for practical RL research.
