  Potential Based Reward Shaping combined with a potential function based on
appropriately defined abstract knowledge has been shown to significantly
improve learning speed in Reinforcement Learning. MultiGrid Reinforcement
Learning (MRL) has further shown that such abstract knowledge in the form of a
potential function can be learned almost solely from agent interaction with the
environment. However, we show that MRL faces the problem of not extending well
to work with Deep Learning. In this paper we extend and improve MRL to take
advantage of modern Deep Learning algorithms such as Deep Q-Networks (DQN). We
show that DQN augmented with our approach perform significantly better on
continuous control tasks than its Vanilla counterpart and DQN augmented with
MRL.
