  This tutorial explains Linear Discriminant Analysis (LDA) and Quadratic
Discriminant Analysis (QDA) as two fundamental classification methods in
statistical and probabilistic learning. We start with the optimization of
decision boundary on which the posteriors are equal. Then, LDA and QDA are
derived for binary and multiple classes. The estimation of parameters in LDA
and QDA are also covered. Then, we explain how LDA and QDA are related to
metric learning, kernel principal component analysis, Mahalanobis distance,
logistic regression, Bayes optimal classifier, Gaussian naive Bayes, and
likelihood ratio test. We also prove that LDA and Fisher discriminant analysis
are equivalent. We finally clarify some of the theoretical concepts with
simulations we provide.
