  Catastrophic forgetting remains a critical challenge in the field of
continual learning, where neural networks struggle to retain prior knowledge
while assimilating new information. Most existing studies emphasize mitigating
this issue only when encountering new tasks, overlooking the significance of
the pre-task phase. Therefore, we shift the attention to the current task
learning stage, presenting a novel framework, C&F (Create and Find Flatness),
which builds a flat training space for each task in advance. Specifically,
during the learning of the current task, our framework adaptively creates a
flat region around the minimum in the loss landscape. Subsequently, it finds
the parameters' importance to the current task based on their flatness degrees.
When adapting the model to a new task, constraints are applied according to the
flatness and a flat space is simultaneously prepared for the impending task. We
theoretically demonstrate the consistency between the created and found
flatness. In this manner, our framework not only accommodates ample parameter
space for learning new tasks but also preserves the preceding knowledge of
earlier tasks. Experimental results exhibit C&F's state-of-the-art performance
as a standalone continual learning approach and its efficacy as a framework
incorporating other methods. Our work is available at
https://github.com/Eric8932/Create-and-Find-Flatness.
