  This study investigates the loss of generalization ability in neural
networks, revisiting warm-starting experiments from Ash & Adams. Our empirical
analysis reveals that common methods designed to enhance plasticity by
maintaining trainability provide limited benefits to generalization. While
reinitializing the network can be effective, it also risks losing valuable
prior knowledge. To this end, we introduce the Hare & Tortoise, inspired by the
brain's complementary learning system. Hare & Tortoise consists of two
components: the Hare network, which rapidly adapts to new information
analogously to the hippocampus, and the Tortoise network, which gradually
integrates knowledge akin to the neocortex. By periodically reinitializing the
Hare network to the Tortoise's weights, our method preserves plasticity while
retaining general knowledge. Hare & Tortoise can effectively maintain the
network's ability to generalize, which improves advanced reinforcement learning
algorithms on the Atari-100k benchmark. The code is available at
https://github.com/dojeon-ai/hare-tortoise.
