  The function approximators employed by traditional image-based Deep
Reinforcement Learning (DRL) algorithms usually lack a temporal learning
component and instead focus on learning the spatial component. We propose a
technique, Temporal Shift Reinforcement Learning (TSRL), wherein both temporal,
as well as spatial components are jointly learned. Moreover, TSRL does not
require additional parameters to perform temporal learning. We show that TSRL
outperforms the commonly used frame stacking heuristic on both of the Atari
environments we test on while beating the SOTA for one of them. This
investigation has implications in the robotics as well as sequential
decision-making domains.
