  Deep reinforcement learning (deep RL) has achieved superior performance in
complex sequential tasks by using deep neural networks as function
approximators to learn directly from raw input images. However, learning
directly from raw images is data inefficient. The agent must learn feature
representation of complex states in addition to learning a policy. As a result,
deep RL typically suffers from slow learning speeds and often requires a
prohibitively large amount of training time and data to reach reasonable
performance, making it inapplicable to real-world settings where data is
expensive. In this work, we improve data efficiency in deep RL by addressing
one of the two learning goals, feature learning. We leverage supervised
learning to pre-train on a small set of non-expert human demonstrations and
empirically evaluate our approach using the asynchronous advantage actor-critic
algorithms (A3C) in the Atari domain. Our results show significant improvements
in learning speed, even when the provided demonstration is noisy and of low
quality.
