  Most reinforcement learning algorithms take advantage of an experience replay
buffer to repeatedly train on samples the agent has observed in the past. Not
all samples carry the same amount of significance and simply assigning equal
importance to each of the samples is a na\"ive strategy. In this paper, we
propose a method to prioritize samples based on how much we can learn from a
sample. We define the learn-ability of a sample as the steady decrease of the
training loss associated with this sample over time. We develop an algorithm to
prioritize samples with high learn-ability, while assigning lower priority to
those that are hard-to-learn, typically caused by noise or stochasticity. We
empirically show that our method is more robust than random sampling and also
better than just prioritizing with respect to the training loss, i.e. the
temporal difference loss, which is used in prioritized experience replay.
