  Deep generative replay has emerged as a promising approach for continual
learning in decision-making tasks. This approach addresses the problem of
catastrophic forgetting by leveraging the generation of trajectories from
previously encountered tasks to augment the current dataset. However, existing
deep generative replay methods for continual learning rely on autoregressive
models, which suffer from compounding errors in the generated trajectories. In
this paper, we propose a simple, scalable, and non-autoregressive method for
continual learning in decision-making tasks using a generative model that
generates task samples conditioned on the trajectory timestep. We evaluate our
method on Continual World benchmarks and find that our approach achieves
state-of-the-art performance on the average success rate metric among continual
learning methods. Code is available at https://github.com/WilliamYue37/t-DGR.
