  We propose a distributed deep learning model to successfully learn control
policies directly from high-dimensional sensory input using reinforcement
learning. The model is based on the deep Q-network, a convolutional neural
network trained with a variant of Q-learning. Its input is raw pixels and its
output is a value function estimating future rewards from taking an action
given a system state. To distribute the deep Q-network training, we adapt the
DistBelief software framework to the context of efficiently training
reinforcement learning agents. As a result, the method is completely
asynchronous and scales well with the number of machines. We demonstrate that
the deep Q-network agent, receiving only the pixels and the game score as
inputs, was able to achieve reasonable success on a simple game with minimal
parameter tuning.
