  We present ContainerGym, a benchmark for reinforcement learning inspired by a
real-world industrial resource allocation task. The proposed benchmark encodes
a range of challenges commonly encountered in real-world sequential decision
making problems, such as uncertainty. It can be configured to instantiate
problems of varying degrees of difficulty, e.g., in terms of variable
dimensionality. Our benchmark differs from other reinforcement learning
benchmarks, including the ones aiming to encode real-world difficulties, in
that it is directly derived from a real-world industrial problem, which
underwent minimal simplification and streamlining. It is sufficiently versatile
to evaluate reinforcement learning algorithms on any real-world problem that
fits our resource allocation framework. We provide results of standard baseline
methods. Going beyond the usual training reward curves, our results and the
statistical tools used to interpret them allow to highlight interesting
limitations of well-known deep reinforcement learning algorithms, namely PPO,
TRPO and DQN.
