  Continual Learning (CL) aims to incrementally update a trained model on new
tasks without forgetting the acquired knowledge of old ones. Existing CL
methods usually reduce forgetting with task priors, \ie using task identity or
a subset of previously seen samples for model training. However, these methods
would be infeasible when such priors are unknown in real-world applications. To
address this fundamental but seldom-studied problem, we propose a Prior-Free
Continual Learning (PFCL) method, which learns new tasks without knowing the
task identity or any previous data. First, based on a fixed single-head
architecture, we eliminate the need for task identity to select the
task-specific output head. Second, we employ a regularization-based strategy
for consistent predictions between the new and old models, avoiding revisiting
previous samples. However, using this strategy alone often performs poorly in
class-incremental scenarios, particularly for a long sequence of tasks. By
analyzing the effectiveness and limitations of conventional
regularization-based methods, we propose enhancing model consistency with an
auxiliary unlabeled dataset additionally. Moreover, since some auxiliary data
may degrade the performance, we further develop a reliable sample selection
strategy to obtain consistent performance improvement. Extensive experiments on
multiple image classification benchmark datasets show that our PFCL method
significantly mitigates forgetting in all three learning scenarios.
Furthermore, when compared to the most recent rehearsal-based methods that
replay a limited number of previous samples, PFCL achieves competitive
accuracy. Our code is available at: https://github.com/visiontao/pfcl
