  Recent advances in mathematical programming have made Mixed Integer
Optimization a competitive alternative to popular regularization methods for
selecting features in regression problems. The approach exhibits unquestionable
foundational appeal and versatility, but also poses important challenges. Here
we propose MIP-BOOST, a revision of standard Mixed Integer Programming feature
selection that reduces the computational burden of tuning the critical sparsity
bound parameter and improves performance in the presence of feature
collinearity and of signals that vary in nature and strength. The final outcome
is a more efficient and effective $L_0$ Feature Selection method for
applications of realistic size and complexity, grounded on rigorous
cross-validation tuning and exact optimization of the associated Mixed Integer
Program. Computational viability and improved performance in realistic
scenarios is achieved through three independent but synergistic proposals.
