  Sparse linear regression methods generally have a free hyperparameter which
controls the amount of sparsity, and is subject to a bias-variance tradeoff.
This article considers the use of Aggregated hold-out to aggregate over values
of this hyperparameter, in the context of linear regression with the Huber loss
function. Aggregated hold-out (Agghoo) is a procedure which averages estimators
selected by hold-out (cross-validation with a single split). In the theoretical
part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle
inequality when it is applied to sparse estimators which are parametrized by
their zero-norm. In particular , this includes a variant of the Lasso
introduced by Zou, Hasti{\'e} and Tibshirani. Simulations are used to compare
Agghoo with cross-validation. They show that Agghoo performs better than CV
when the intrinsic dimension is high and when there are confounders correlated
with the predictive covariates.
