DeepAI AI Chat
Log In Sign Up

Best-first Search Algorithm for Non-convex Sparse Minimization

by   Shinsaku Sakaue, et al.

Non-convex sparse minimization (NSM), or ℓ_0-constrained minimization of convex loss functions, is an important optimization problem that has many machine learning applications. NSM is generally NP-hard, and so to exactly solve NSM is almost impossible in polynomial time. As regards the case of quadratic objective functions, exact algorithms based on quadratic mixed-integer programming (MIP) have been studied, but no existing exact methods can handle more general objective functions including Huber and logistic losses; this is unfortunate since those functions are prevalent in practice. In this paper, we consider NSM with ℓ_2-regularized convex objective functions and develop an algorithm by leveraging the efficiency of best-first search (BFS). Our BFS can compute solutions with objective errors at most Δ>0, where Δ is a controllable hyper-parameter that balances the trade-off between the guarantee of objective errors and computation cost. Experiments demonstrate that our BFS is useful for solving moderate-size NSM instances with non-quadratic objectives and that BFS is also faster than the MIP-based method when applied to quadratic objectives.


page 1

page 2

page 3

page 4


Convexification of Learning from Constraints

Regularized empirical risk minimization with constrained labels (in cont...

Proximal Algorithms in Statistics and Machine Learning

In this paper we develop proximal methods for statistical learning. Prox...

Fast Large-Scale Discrete Optimization Based on Principal Coordinate Descent

Binary optimization, a representative subclass of discrete optimization,...

Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features

The Frank-Wolfe (FW) method is a popular approach for solving optimizati...

Challenges of Convex Quadratic Bi-objective Benchmark Problems

Convex quadratic objective functions are an important base case in state...

Generalizing Trajectory Retiming to Quadratic Objective Functions

Trajectory retiming is the task of computing a feasible time parameteriz...

Multiscale semidefinite programming approach to positioning problems with pairwise structure

We consider the optimization of pairwise objective functions, i.e., obje...