  Learning unbiased models on imbalanced datasets is a significant challenge.
Rare classes tend to get a concentrated representation in the classification
space which hampers the generalization of learned boundaries to new test
examples. In this paper, we demonstrate that the Bayesian uncertainty estimates
directly correlate with the rarity of classes and the difficulty level of
individual samples. Subsequently, we present a novel framework for uncertainty
based class imbalance learning that follows two key insights: First,
classification boundaries should be extended further away from a more uncertain
(rare) class to avoid overfitting and enhance its generalization. Second, each
sample should be modeled as a multi-variate Gaussian distribution with a mean
vector and a covariance matrix defined by the sample's uncertainty. The learned
boundaries should respect not only the individual samples but also their
distribution in the feature space. Our proposed approach efficiently utilizes
sample and class uncertainty information to learn robust features and more
generalizable classifiers. We systematically study the class imbalance problem
and derive a novel loss formulation for max-margin learning based on Bayesian
uncertainty measure. The proposed method shows significant performance
improvements on six benchmark datasets for face verification, attribute
prediction, digit/object classification and skin lesion detection.
