Conditional Gradient Algorithms for Norm-Regularized Smooth Convex Optimization

02/10/2013
by   Zaid Harchaoui, et al.
0

Motivated by some applications in signal processing and machine learning, we consider two convex optimization problems where, given a cone K, a norm · and a smooth convex function f, we want either 1) to minimize the norm over the intersection of the cone and a level set of f, or 2) to minimize over the cone the sum of f and a multiple of the norm. We focus on the case where (a) the dimension of the problem is too large to allow for interior point algorithms, (b) · is "too complicated" to allow for computationally cheap Bregman projections required in the first-order proximal gradient algorithms. On the other hand, we assume that it is relatively easy to minimize linear forms over the intersection of K and the unit ·-ball. Motivating examples are given by the nuclear norm with K being the entire space of matrices, or the positive semidefinite cone in the space of symmetric matrices, and the Total Variation norm on the space of 2D images. We discuss versions of the Conditional Gradient algorithm capable to handle our problems of interest, provide the related theoretical efficiency estimates and outline some applications.

READ FULL TEXT
research
04/14/2014

Hybrid Conditional Gradient - Smoothing Algorithms with Applications to Sparse and Low Rank Regularization

We study a hybrid conditional gradient - smoothing algorithm (HCGS) for ...
research
02/10/2014

Signal Reconstruction Framework Based On Projections Onto Epigraph Set Of A Convex Cost Function (PESC)

A new signal processing framework based on making orthogonal Projections...
research
05/01/2010

Perturbation Resilience and Superiorization of Iterative Algorithms

Iterative algorithms aimed at solving some problems are discussed. For c...
research
08/04/2023

Adaptive Proximal Gradient Method for Convex Optimization

In this paper, we explore two fundamental first-order algorithms in conv...
research
02/12/2021

From perspective maps to epigraphical projections

The projection onto the epigraph or a level set of a closed proper conve...
research
04/05/2017

Geometry of Factored Nuclear Norm Regularization

This work investigates the geometry of a nonconvex reformulation of mini...
research
12/14/2020

On the Treatment of Optimization Problems with L1 Penalty Terms via Multiobjective Continuation

We present a novel algorithm that allows us to gain detailed insight int...

Please sign up or login with your details

Forgot password? Click here to reset