  Model Weight Averaging (MWA) is a technique that seeks to enhance model's
performance by averaging the weights of multiple trained models. This paper
first empirically finds that 1) the vanilla MWA can benefit the
class-imbalanced learning, and 2) performing model averaging in the early
epochs of training yields a greater performance improvement than doing that in
later epochs. Inspired by these two observations, in this paper we propose a
novel MWA technique for class-imbalanced learning tasks named Iterative Model
Weight Averaging (IMWA). Specifically, IMWA divides the entire training stage
into multiple episodes. Within each episode, multiple models are concurrently
trained from the same initialized model weight, and subsequently averaged into
a singular model. Then, the weight of this average model serves as a fresh
initialization for the ensuing episode, thus establishing an iterative learning
paradigm. Compared to vanilla MWA, IMWA achieves higher performance
improvements with the same computational cost. Moreover, IMWA can further
enhance the performance of those methods employing EMA strategy, demonstrating
that IMWA and EMA can complement each other. Extensive experiments on various
class-imbalanced learning tasks, i.e., class-imbalanced image classification,
semi-supervised class-imbalanced image classification and semi-supervised
object detection tasks showcase the effectiveness of our IMWA.
