  Experience replay (ER) is a crucial component of many deep reinforcement
learning (RL) systems. However, uniform sampling from an ER buffer can lead to
slow convergence and unstable asymptotic behaviors. This paper introduces
Stratified Sampling from Event Tables (SSET), which partitions an ER buffer
into Event Tables, each capturing important subsequences of optimal behavior.
We prove a theoretical advantage over the traditional monolithic buffer
approach and combine SSET with an existing prioritized sampling strategy to
further improve learning speed and stability. Empirical results in challenging
MiniGrid domains, benchmark RL environments, and a high-fidelity car racing
simulator demonstrate the advantages and versatility of SSET over existing ER
buffer sampling approaches.
