  We investigate the difference between using an $\ell_1$ penalty versus an
$\ell_1$ constraint in generalized eigenvalue problems, such as principal
component analysis and discriminant analysis. Our main finding is that an
$\ell_1$ penalty may fail to provide very sparse solutions; a severe
disadvantage for variable selection that can be remedied by using an $\ell_1$
constraint. Our claims are supported both by empirical evidence and theoretical
analysis. Finally, we illustrate the advantages of an $\ell_1$ constraint in
the context of discriminant analysis and principal component analysis.
