  Reinforcement learning (RL) has been successful in training agents in various
learning environments, including video-games. However, such work modifies and
shrinks the action space from the game's original. This is to avoid trying
"pointless" actions and to ease the implementation. Currently, this is mostly
done based on intuition, with little systematic research supporting the design
decisions. In this work, we aim to gain insight on these action space
modifications by conducting extensive experiments in video-game environments.
Our results show how domain-specific removal of actions and discretization of
continuous actions can be crucial for successful learning. With these insights,
we hope to ease the use of RL in new environments, by clarifying what
action-spaces are easy to learn.
