  The adaptive lasso refers to a class of methods that use weighted versions of
the $L_1$-norm penalty, with weights derived from an initial estimate of the
parameter vector to be estimated. Irrespective of the method chosen to compute
this initial estimate, the performance of the adaptive lasso critically depends
on the value of a hyperparameter, which controls the magnitude of the weighted
$L_1$-norm in the penalized criterion. As for other machine learning methods,
cross-validation is very popular for the calibration of the adaptive lasso,
that this for the selection of a data-driven optimal value of this
hyperparameter. However, the most simple cross-validation scheme is not valid
in this context, and a more elaborate one has to be employed to guarantee an
optimal calibration. The discrepancy of the simple cross-validation scheme has
been well documented in other contexts, but less so when it comes to the
calibration of the adaptive lasso, and, therefore, many statistical analysts
still overlook it. In this work, we recall appropriate cross-validation schemes
for the calibration of the adaptive lasso, and illustrate the discrepancy of
the simple scheme, using both synthetic and real-world examples. Our results
clearly establish the suboptimality of the simple scheme, in terms of support
recovery and prediction error, for several versions of the adaptive lasso,
including the popular one-step lasso.
