  Least squares fitting is in general not useful for high-dimensional linear
models, in which the number of predictors is of the same or even larger order
of magnitude than the number of samples. Theory developed in recent years has
coined a paradigm according to which sparsity-promoting regularization is
regarded as a necessity in such setting. Deviating from this paradigm, we show
that non-negativity constraints on the regression coefficients may be similarly
effective as explicit regularization if the design matrix has additional
properties, which are met in several applications of non-negative least squares
(NNLS). We show that for these designs, the performance of NNLS with regard to
prediction and estimation is comparable to that of the lasso. We argue further
that in specific cases, NNLS may have a better $\ell_{\infty}$-rate in
estimation and hence also advantages with respect to support recovery when
combined with thresholding. From a practical point of view, NNLS does not
depend on a regularization parameter and is hence easier to use.
