  Contamination can severely distort an estimator unless the estimation
procedure is suitably robust. This is a well-known issue and has been addressed
in Robust Statistics, however, the relation of contamination and distorted
variable selection has been rarely considered in literature. As for variable
selection, many methods for sparse model selection have been proposed,
including the Stability Selection which is a meta-algorithm based on some
variable selection algorithm in order to immunize against particular data
configurations. We introduce the variable selection breakdown point that
quantifies the number of cases resp. cells that have to be contaminated in
order to let no relevant variable be detected. We show that particular outlier
configurations can completely mislead model selection and argue why even
cell-wise robust methods cannot fix this problem. We combine the variable
selection breakdown point with resampling, resulting in the Stability Selection
breakdown point that quantifies the robustness of Stability Selection. We
propose a trimmed Stability Selection which only aggregates the models with the
lowest in-sample losses so that, heuristically, models computed on heavily
contaminated resamples should be trimmed away. An extensive simulation study
with non-robust regression and classification algorithms as well as with Sparse
Least Trimmed Squares reveals both the potential of our approach to boost the
model selection robustness as well as the fragility of variable selection using
non-robust algorithms, even for an extremely small cell-wise contamination
rate.
