Homotopy Parametric Simplex Method for Sparse Learning

04/04/2017
by   Haotian Pang, et al.
0

High dimensional sparse learning has imposed a great computational challenge to large scale data analysis. In this paper, we are interested in a broad class of sparse learning approaches formulated as linear programs parametrized by a regularization factor, and solve them by the parametric simplex method (PSM). Our parametric simplex method offers significant advantages over other competing methods: (1) PSM naturally obtains the complete solution path for all values of the regularization parameter; (2) PSM provides a high precision dual certificate stopping criterion; (3) PSM yields sparse solutions through very few iterations, and the solution sparsity significantly reduces the computational cost per iteration. Particularly, we demonstrate the superiority of PSM over various sparse learning approaches, including Dantzig selector for sparse linear regression, LAD-Lasso for sparse robust linear regression, CLIME for sparse precision matrix estimation, sparse differential network estimation, and sparse Linear Programming Discriminant (LPD) analysis. We then provide sufficient conditions under which PSM always outputs sparse solutions such that its computational performance can be significantly boosted. Thorough numerical experiments are provided to demonstrate the outstanding performance of the PSM method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/12/2019

Sparse estimation via ℓ_q optimization method in high-dimensional linear regression

In this paper, we discuss the statistical properties of the ℓ_q optimiza...
research
06/27/2020

The flare Package for High Dimensional Linear Regression and Precision Matrix Estimation in R

This paper describes an R package named flare, which implements a family...
research
03/22/2019

Implicit Regularization via Hadamard Product Over-Parametrization in High-Dimensional Linear Regression

We consider Hadamard product parametrization as a change-of-variable (ov...
research
05/16/2020

Nested Model Averaging on Solution Path for High-dimensional Linear Regression

We study the nested model averaging method on the solution path for a hi...
research
10/29/2019

Efficient Computation for Centered Linear Regression with Sparse Inputs

Regression with sparse inputs is a common theme for large scale models. ...
research
07/10/2019

Sparse Unit-Sum Regression

This paper considers sparsity in linear regression under the restriction...
research
03/14/2022

Mixture Components Inference for Sparse Regression: Introduction and Application for Estimation of Neuronal Signal from fMRI BOLD

Sparse linear regression methods including the well-known LASSO and the ...

Please sign up or login with your details

Forgot password? Click here to reset