  Parameter regularization or allocation methods are effective in overcoming
catastrophic forgetting in lifelong learning. However, they solve all tasks in
a sequence uniformly and ignore the differences in the learning difficulty of
different tasks. So parameter regularization methods face significant
forgetting when learning a new task very different from learned tasks, and
parameter allocation methods face unnecessary parameter overhead when learning
simple tasks. In this paper, we propose the Parameter Allocation &
Regularization (PAR), which adaptively select an appropriate strategy for each
task from parameter allocation and regularization based on its learning
difficulty. A task is easy for a model that has learned tasks related to it and
vice versa. We propose a divergence estimation method based on the
Nearest-Prototype distance to measure the task relatedness using only features
of the new task. Moreover, we propose a time-efficient relatedness-aware
sampling-based architecture search strategy to reduce the parameter overhead
for allocation. Experimental results on multiple benchmarks demonstrate that,
compared with SOTAs, our method is scalable and significantly reduces the
model's redundancy while improving the model's performance. Further qualitative
analysis indicates that PAR obtains reasonable task-relatedness.
