  Model selection consistency in the high-dimensional regression setting can be
achieved only if strong assumptions are fulfilled. We therefore suggest to
pursue a different goal, which we call a minimal class of models. The minimal
class of models includes models that are similar in their prediction accuracy
but not necessarily in their elements. We suggest a random search algorithm to
reveal candidate models. The algorithm implements simulated annealing while
using a score for each predictor that we suggest to derive using a combination
of the Lasso and the Elastic Net. The utility of using a minimal class of
models is demonstrated in the analysis of two datasets.
