Generalized Conjugate Gradient Methods for ℓ_1 Regularized Convex Quadratic Programming with Finite Convergence

by   Zhaosong Lu, et al.

The conjugate gradient (CG) method is an efficient iterative method for solving large-scale strongly convex quadratic programming (QP). In this paper we propose some generalized CG (GCG) methods for solving the ℓ_1-regularized (possibly not strongly) convex QP that terminate at an optimal solution in a finite number of iterations. At each iteration, our methods first identify a face of an orthant and then either perform an exact line search along the direction of the negative projected minimum-norm subgradient of the objective function or execute a CG subroutine that conducts a sequence of CG iterations until a CG iterate crosses the boundary of this face or an approximate minimizer of over this face or a subface is found. We determine which type of step should be taken by comparing the magnitude of some components of the minimum-norm subgradient of the objective function to that of its rest components. Our analysis on finite convergence of these methods makes use of an error bound result and some key properties of the aforementioned exact line search and the CG subroutine. We also show that the proposed methods are capable of finding an approximate solution of the problem by allowing some inexactness on the execution of the CG subroutine. The overall arithmetic operation cost of our GCG methods for finding an ϵ-optimal solution depends on ϵ in O((1/ϵ)), which is superior to the accelerated proximal gradient method [2,23] that depends on ϵ in O(1/√(ϵ)). In addition, our GCG methods can be extended straightforwardly to solve box-constrained convex QP with finite convergence. Numerical results demonstrate that our methods are very favorable for solving ill-conditioned problems.


page 1

page 2

page 3

page 4


Iterative Hard Thresholding Methods for l_0 Regularized Convex Cone Programming

In this paper we consider l_0 regularized convex cone programming proble...

Using a New Nonlinear Gradient Method for Solving Large Scale Convex Optimization Problems with an Application on Arabic Medical Text

Gradient methods have applications in multiple fields, including signal ...

A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem

We consider solving the ℓ_1-regularized least-squares (ℓ_1-LS) problem i...

The convergence of the Generalized Lanczos Trust-Region Method for the Trust-Region Subproblem

Solving the trust-region subproblem (TRS) plays a key role in numerical ...

Revisiting Frank-Wolfe for Polytopes: Strict Complementary and Sparsity

In recent years it was proved that simple modifications of the classical...

A twin error gauge for Kaczmarz's iterations

We propose two new methods based on Kaczmarz's method that produce a reg...

Please sign up or login with your details

Forgot password? Click here to reset