  We present local discriminative Gaussian (LDG) dimensionality reduction, a
supervised dimensionality reduction technique for classification. The LDG
objective function is an approximation to the leave-one-out training error of a
local quadratic discriminant analysis classifier, and thus acts locally to each
training point in order to find a mapping where similar data can be
discriminated from dissimilar data. While other state-of-the-art linear
dimensionality reduction methods require gradient descent or iterative solution
approaches, LDG is solved with a single eigen-decomposition. Thus, it scales
better for datasets with a large number of feature dimensions or training
examples. We also adapt LDG to the transfer learning setting, and show that it
achieves good performance when the test data distribution differs from that of
the training data.
