Penalized versus constrained generalized eigenvalue problems

10/22/2014

∙

by   Irina Gaynanova, et al.

∙

0

∙

share

We investigate the difference between using an ℓ_1 penalty versus an ℓ_1 constraint in generalized eigenvalue problems, such as principal component analysis and discriminant analysis. Our main finding is that an ℓ_1 penalty may fail to provide very sparse solutions; a severe disadvantage for variable selection that can be remedied by using an ℓ_1 constraint. Our claims are supported both by empirical evidence and theoretical analysis. Finally, we illustrate the advantages of an ℓ_1 constraint in the context of discriminant analysis and principal component analysis.