  The application of standard sufficient dimension reduction methods for
reducing the dimension space of predictors without losing regression
information requires inverting the covariance matrix of the predictors. This
has posed a number of challenges especially when analyzing high-dimensional
data sets in which the number of predictors $\mathit{p}$ is much larger than
number of samples $n,~(n\ll p)$. A new covariance estimator, called the
\textit{Maximum Entropy Covariance} (MEC) that addresses loss of covariance
information when similar covariance matrices are linearly combined using
\textit{Maximum Entropy} (ME) principle is proposed in this work. By
benefitting naturally from slicing or discretizing range of the response
variable, y into \textit{H} non-overlapping categories, $\mathit{h_{1},\ldots
,h_{H}}$, MEC first combines covariance matrices arising from samples in each y
slice $\mathit{h\in H}$ and then select the one that maximizes entropy under
the principle of maximum uncertainty. The MEC estimator is then formed from
convex mixture of such entropy-maximizing sample covariance $S_{\mbox{mec}}$
estimate and pooled sample covariance $\mathbf{S}_{\mathit{p}}$ estimate across
the $\mathit{H}$ slices without requiring time-consuming covariance
optimization procedures. MEC deals directly with singularity and instability of
sample group covariance estimate in both regression and classification
problems. The efficiency of the MEC estimator is studied with the existing
sufficient dimension reduction methods such as \textit{Sliced Inverse
Regression} (SIR) and \textit{Sliced Average Variance Estimator} (SAVE) as
demonstrated on both classification and regression problems using real life
Leukemia cancer data and customers' electricity load profiles from smart meter
data sets respectively.
