  Recent advancements in Mixed Integer Optimization (MIO) algorithms, paired
with hardware enhancements, have led to significant speedups in resolving MIO
problems. These strategies have been utilized for optimal subset selection,
specifically for choosing $k$ features out of $p$ in linear regression given
$n$ observations. In this paper, we broaden this method to facilitate
cluster-aware regression, where selection aims to choose $\lambda$ out of $K$
clusters in a linear mixed effects (LMM) model with $n_k$ observations for each
cluster. Through comprehensive testing on a multitude of synthetic and real
datasets, we exhibit that our method efficiently solves problems within
minutes. Through numerical experiments, we also show that the MIO approach
outperforms both Gaussian- and Laplace-distributed LMMs in terms of generating
sparse solutions with high predictive power. Traditional LMMs typically assume
that clustering effects are independent of individual features. However, we
introduce an innovative algorithm that evaluates cluster effects for new data
points, thereby increasing the robustness and precision of this model. The
inferential and predictive efficacy of this approach is further illustrated
through its application in student scoring and protein expression.
