  Pothole classification has become an important task for road inspection
vehicles to save drivers from potential car accidents and repair bills. Given
the limited computational power and fixed number of training epochs, we propose
iterative self knowledge distillation (ISKD) to train lightweight pothole
classifiers. Designed to improve both the teacher and student models over time
in knowledge distillation, ISKD outperforms the state-of-the-art self knowledge
distillation method on three pothole classification datasets across four
lightweight network architectures, which supports that self knowledge
distillation should be done iteratively instead of just once. The accuracy
relation between the teacher and student models shows that the student model
can still benefit from a moderately trained teacher model. Implying that better
teacher models generally produce better student models, our results justify the
design of ISKD. In addition to pothole classification, we also demonstrate the
efficacy of ISKD on six additional datasets associated with generic
classification, fine-grained classification, and medical imaging application,
which supports that ISKD can serve as a general-purpose performance booster
without the need of a given teacher model and extra trainable parameters.
