  Recent advancements in deep learning have shifted the development of brain
imaging analysis. However, several challenges remain, such as heterogeneity,
individual variations, and the contradiction between the high dimensionality
and small size of brain imaging datasets. These issues complicate the learning
process, preventing models from capturing intrinsic, meaningful patterns and
potentially leading to suboptimal performance due to biases and overfitting.
Curriculum learning (CL) presents a promising solution by organizing training
examples from simple to complex, mimicking the human learning process, and
potentially fostering the development of more robust and accurate models.
Despite its potential, the inherent limitations posed by small initial training
datasets present significant challenges, including overfitting and poor
generalization. In this paper, we introduce the Progressive Self-Paced
Distillation (PSPD) framework, employing an adaptive and progressive pacing and
distillation mechanism. This allows for dynamic curriculum adjustments based on
the states of both past and present models. The past model serves as a teacher,
guiding the current model with gradually refined curriculum knowledge and
helping prevent the loss of previously acquired knowledge. We validate PSPD's
efficacy and adaptability across various convolutional neural networks using
the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, underscoring
its superiority in enhancing model performance and generalization capabilities.
The source code for this approach will be released at
https://github.com/Hrychen7/PSPD.
