  In this paper, we propose Rogue-Gym, a simple and classic style roguelike
game built for evaluating generalization in reinforcement learning (RL).
Combined with the recent progress of deep neural networks, RL has successfully
trained human-level agents without human knowledge in many games such as those
for Atari 2600. However, it has been pointed out that agents trained with RL
methods often overfit the training environment, and they work poorly in
slightly different environments. To investigate this problem, some research
environments with procedural content generation have been proposed. Following
these studies, we propose the use of roguelikes as a benchmark for evaluating
the generalization ability of RL agents. In our Rogue-Gym, agents need to
explore dungeons that are structured differently each time they start a new
game. Thanks to the very diverse structures of the dungeons, we believe that
the generalization benchmark of Rogue-Gym is sufficiently fair. In our
experiments, we evaluate a standard reinforcement learning method, PPO, with
and without enhancements for generalization. The results show that some
enhancements believed to be effective fail to mitigate the overfitting in
Rogue-Gym, although others slightly improve the generalization ability.
