  This paper considers deep visual recognition on long-tailed data. To be
general, we consider two applied scenarios, \ie, deep classification and deep
metric learning. Under the long-tailed data distribution, the majority classes
(\ie, tail classes) only occupy relatively few samples and are prone to lack of
within-class diversity. A radical solution is to augment the tail classes with
higher diversity. To this end, we introduce a simple and reliable method named
Memory-based Jitter (MBJ). We observe that during training, the deep model
constantly changes its parameters after every iteration, yielding the
phenomenon of \emph{weight jitters}. Consequentially, given a same image as the
input, two historical editions of the model generate two different features in
the deeply-embedded space, resulting in \emph{feature jitters}. Using a memory
bank, we collect these (model or feature) jitters across multiple training
iterations and get the so-called Memory-based Jitter. The accumulated jitters
enhance the within-class diversity for the tail classes and consequentially
improves long-tailed visual recognition. With slight modifications, MBJ is
applicable for two fundamental visual recognition tasks, \emph{i.e.}, deep
image classification and deep metric learning (on long-tailed data). Extensive
experiments on five long-tailed classification benchmarks and two deep metric
learning benchmarks demonstrate significant improvement. Moreover, the achieved
performance are on par with the state of the art on both tasks.
