  This work introduces a refinement of the Parsimonious Model for fitting a
Gaussian Mixture. The improvement is based on the consideration of clusters of
the involved covariance matrices according to a criterion, such as sharing
Principal Directions. This and other similarity criteria that arise from the
spectral decomposition of a matrix are the bases of the Parsimonious Model. We
show that such groupings of covariance matrices can be achieved through simple
modifications of the CEM (Classification Expectation Maximization) algorithm.
Our approach leads to propose Gaussian Mixture Models for model-based
clustering and discriminant analysis, in which covariance matrices are
clustered according to a parsimonious criterion, creating intermediate steps
between the fourteen widely known parsimonious models. The added versatility
not only allows us to obtain models with fewer parameters for fitting the data,
but also provides greater interpretability. We show its usefulness for
model-based clustering and discriminant analysis, providing algorithms to find
approximate solutions verifying suitable size, shape and orientation
constraints, and applying them to both simulation and real data examples.
