  We consider the problem of finding tuned regularized parameter estimators for
linear models. We start by showing that three known optimal linear estimators
belong to a wider class of estimators that can be formulated as a solution to a
weighted and constrained minimization problem. The optimal weights, however,
are typically unknown in many applications. This begs the question, how should
we choose the weights using only the data? We propose using the covariance
fitting SPICE-methodology to obtain data-adaptive weights and show that the
resulting class of estimators yields tuned versions of known regularized
estimators - such as ridge regression, LASSO, and regularized least absolute
deviation. These theoretical results unify several important estimators under a
common umbrella. The resulting tuned estimators are also shown to be
practically relevant by means of a number of numerical examples.
