  Convolutional neural networks (CNNs) have achieved impressive results on
imbalanced image data, but they still have difficulty generalizing to minority
classes and their decisions are difficult to interpret. These problems are
related because the method by which CNNs generalize to minority classes, which
requires improvement, is wrapped in a blackbox. To demystify CNN decisions on
imbalanced data, we focus on their latent features. Although CNNs embed the
pattern knowledge learned from a training set in model parameters, the effect
of this knowledge is contained in feature and classification embeddings (FE and
CE). These embeddings can be extracted from a trained model and their global,
class properties (e.g., frequency, magnitude and identity) can be analyzed. We
find that important information regarding the ability of a neural network to
generalize to minority classes resides in the class top-K CE and FE. We show
that a CNN learns a limited number of class top-K CE per category, and that
their number and magnitudes vary based on whether the same class is balanced or
imbalanced. This calls into question whether a CNN has learned intrinsic class
features, or merely frequently occurring ones that happen to exist in the
sampled class distribution. We also hypothesize that latent class diversity is
as important as the number of class examples, which has important implications
for re-sampling and cost-sensitive methods. These methods generally focus on
rebalancing model weights, class numbers and margins; instead of diversifying
class latent features through augmentation. We also demonstrate that a CNN has
difficulty generalizing to test data if the magnitude of its top-K latent
features do not match the training set. We use three popular image datasets and
two cost-sensitive algorithms commonly employed in imbalanced learning for our
experiments.
