  In this work, we characterize two data piling phenomenon for a
high-dimensional binary classification problem with heterogeneous covariance
models. The data piling refers to the phenomenon where projections of the
training data onto a direction vector have exactly two distinct values, one for
each class. This first data piling phenomenon occurs for any data when the
dimension $p$ is larger than the sample size $n$. We show that the second data
piling phenomenon, which refers to a data piling of independent test data, can
occur in an asymptotic context where $p$ grows while $n$ is fixed. We further
show that a second maximal data piling direction, which gives an asymptotic
maximal distance between the two piles of independent test data, can be
obtained by projecting the first maximal data piling direction onto the
nullspace of the common leading eigenspace. This observation provides a
theoretical explanation for the phenomenon where the optimal ridge parameter
can be negative in the context of high-dimensional linear classification. Based
on the second data piling phenomenon, we propose various linear classification
rules which ensure perfect classification of high-dimension low-sample-size
data under generalized heterogeneous spiked covariance models.
