  The ability of continual learning systems to transfer knowledge from
previously seen tasks in order to maximize performance on new tasks is a
significant challenge for the field, limiting the applicability of continual
learning solutions to realistic scenarios. Consequently, this study aims to
broaden our understanding of transfer and its driving forces in the specific
case of continual reinforcement learning. We adopt SAC as the underlying RL
algorithm and Continual World as a suite of continuous control tasks. We
systematically study how different components of SAC (the actor and the critic,
exploration, and data) affect transfer efficacy, and we provide recommendations
regarding various modeling options. The best set of choices, dubbed ClonEx-SAC,
is evaluated on the recent Continual World benchmark. ClonEx-SAC achieves 87%
final success rate compared to 80% of PackNet, the best method in the
benchmark. Moreover, the transfer grows from 0.18 to 0.54 according to the
metric provided by Continual World.
