A Scalable and Extensible Framework for Superposition-Structured Models

09/08/2015
by   Shenjian Zhao, et al.
0

In many learning tasks, structural models usually lead to better interpretability and higher generalization performance. In recent years, however, the simple structural models such as lasso are frequently proved to be insufficient. Accordingly, there has been a lot of work on "superposition-structured" models where multiple structural constraints are imposed. To efficiently solve these "superposition-structured" statistical models, we develop a framework based on a proximal Newton-type method. Employing the smoothed conic dual approach with the LBFGS updating formula, we propose a scalable and extensible proximal quasi-Newton (SEP-QN) framework. Empirical analysis on various datasets shows that our framework is potentially powerful, and achieves super-linear convergence rate for optimizing some popular "superposition-structured" statistical models such as the fused sparse group lasso.

READ FULL TEXT
research
02/14/2012

Smoothing Proximal Gradient Method for General Structured Sparse Learning

We study the problem of learning high dimensional regression models regu...
research
10/13/2010

Online Multiple Kernel Learning for Structured Prediction

Despite the recent progress towards efficient multiple kernel learning (...
research
06/26/2011

A General Framework for Structured Sparsity via Proximal Optimization

We study a generalized framework for structured sparsity. It extends the...
research
01/08/2013

A proximal Newton framework for composite minimization: Graph learning without Cholesky decompositions and matrix inversions

We propose an algorithmic framework for convex minimization problems of ...
research
08/22/2018

Efficient sparse Hessian based algorithms for the clustered lasso problem

We focus on solving the clustered lasso problem, which is a least square...
research
06/27/2014

Proximal Quasi-Newton for Computationally Intensive L1-regularized M-estimators

We consider the class of optimization problems arising from computationa...
research
02/13/2023

Efficient Graph Laplacian Estimation by a Proximal Newton Approach

The Laplacian-constrained Gaussian Markov Random Field (LGMRF) is a comm...

Please sign up or login with your details

Forgot password? Click here to reset