  The recent advances in deep neural networks have led to effective
vision-based reinforcement learning methods that have been employed to obtain
human-level controllers in Atari 2600 games from pixel data. Atari 2600 games,
however, do not resemble real-world tasks since they involve non-realistic 2D
environments and the third-person perspective. Here, we propose a novel
test-bed platform for reinforcement learning research from raw visual
information which employs the first-person perspective in a semi-realistic 3D
world. The software, called ViZDoom, is based on the classical first-person
shooter video game, Doom. It allows developing bots that play the game using
the screen buffer. ViZDoom is lightweight, fast, and highly customizable via a
convenient mechanism of user scenarios. In the experimental part, we test the
environment by trying to learn bots for two scenarios: a basic move-and-shoot
task and a more complex maze-navigation problem. Using convolutional deep
neural networks with Q-learning and experience replay, for both scenarios, we
were able to train competent bots, which exhibit human-like behaviors. The
results confirm the utility of ViZDoom as an AI research platform and imply
that visual reinforcement learning in 3D realistic first-person perspective
environments is feasible.
