  Linear Discriminant Analysis (LDA) is a widely-used supervised dimensionality
reduction method in computer vision and pattern recognition. In null space
based LDA (NLDA), a well-known LDA extension, between-class distance is
maximized in the null space of the within-class scatter matrix. However, there
are some limitations in NLDA. Firstly, for many data sets, null space of
within-class scatter matrix does not exist, thus NLDA is not applicable to
those datasets. Secondly, NLDA uses arithmetic mean of between-class distances
and gives equal consideration to all between-class distances, which makes
larger between-class distances can dominate the result and thus limits the
performance of NLDA. In this paper, we propose a harmonic mean based Linear
Discriminant Analysis, Multi-Class Discriminant Analysis (MCDA), for image
classification, which minimizes the reciprocal of weighted harmonic mean of
pairwise between-class distance. More importantly, MCDA gives higher priority
to maximize small between-class distances. MCDA can be extended to multi-label
dimension reduction. Results on 7 single-label data sets and 4 multi-label data
sets show that MCDA has consistently better performance than 10 other
single-label approaches and 4 other multi-label approaches in terms of
classification accuracy, macro and micro average F1 score.
