  Data for face analysis often exhibit highly-skewed class distribution, i.e.,
most data belong to a few majority classes, while the minority classes only
contain a scarce amount of instances. To mitigate this issue, contemporary deep
learning methods typically follow classic strategies such as class re-sampling
or cost-sensitive training. In this paper, we conduct extensive and systematic
experiments to validate the effectiveness of these classic schemes for
representation learning on class-imbalanced data. We further demonstrate that
more discriminative deep representation can be learned by enforcing a deep
network to maintain inter-cluster margins both within and between classes. This
tight constraint effectively reduces the class imbalance inherent in the local
data neighborhood, thus carving much more balanced class boundaries locally. We
show that it is easy to deploy angular margins between the cluster
distributions on a hypersphere manifold. Such learned Cluster-based Large
Margin Local Embedding (CLMLE), when combined with a simple k-nearest cluster
algorithm, shows significant improvements in accuracy over existing methods on
both face recognition and face attribute prediction tasks that exhibit
imbalanced class distribution.
