A multilevel framework for sparse optimization with application to inverse covariance estimation and logistic regression

07/01/2016
by   Eran Treister, et al.
0

Solving l1 regularized optimization problems is common in the fields of computational biology, signal processing and machine learning. Such l1 regularization is utilized to find sparse minimizers of convex functions. A well-known example is the LASSO problem, where the l1 norm regularizes a quadratic function. A multilevel framework is presented for solving such l1 regularized sparse optimization problems efficiently. We take advantage of the expected sparseness of the solution, and create a hierarchy of problems of similar type, which is traversed in order to accelerate the optimization process. This framework is applied for solving two problems: (1) the sparse inverse covariance estimation problem, and (2) l1-regularized logistic regression. In the first problem, the inverse of an unknown covariance matrix of a multivariate normal distribution is estimated, under the assumption that it is sparse. To this end, an l1 regularized log-determinant optimization problem needs to be solved. This task is challenging especially for large-scale datasets, due to time and memory limitations. In the second problem, the l1-regularization is added to the logistic regression classification objective to reduce overfitting to the data and obtain a sparse model. Numerical experiments demonstrate the efficiency of the multilevel framework in accelerating existing iterative solvers for both of these problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/24/2014

Distributed Coordinate Descent for L1-regularized Logistic Regression

Solving logistic regression with L1-regularization in distributed settin...
research
03/27/2013

Efficiently Using Second Order Information in Large l1 Regularization Problems

We propose a novel general algorithm LHAC that efficiently uses second-o...
research
11/12/2012

Iterative Thresholding Algorithm for Sparse Inverse Covariance Estimation

The L1-regularized maximum likelihood estimation problem has recently be...
research
06/15/2018

Multilevel preconditioning for Ridge Regression

Solving linear systems is often the computational bottleneck in real-lif...
research
02/26/2021

Sparse Approximations with Interior Point Methods

Large-scale optimization problems that seek sparse solutions have become...
research
04/23/2021

Encrypted Distributed Lasso for Sparse Data Predictive Control

The least squares problem with L1-regularized regressors, called Lasso, ...
research
01/10/2019

Harnessing the Power of Serverless Runtimes for Large-Scale Optimization

The event-driven and elastic nature of serverless runtimes makes them a ...

Please sign up or login with your details

Forgot password? Click here to reset