Penalized versus constrained generalized eigenvalue problems
share
We investigate the difference between using an ℓ_1 penalty versus an ℓ_1 constraint in generalized eigenvalue problems, such as principal component analysis and discriminant analysis. Our main finding is that an ℓ_1 penalty may fail to provide very sparse solutions; a severe disadvantage for variable selection that can be remedied by using an ℓ_1 constraint. Our claims are supported both by empirical evidence and theoretical analysis. Finally, we illustrate the advantages of an ℓ_1 constraint in the context of discriminant analysis and principal component analysis.
READ FULL TEXT