DeepAI AI Chat
Log In Sign Up

Convexification of Learning from Constraints

by   Iaroslav Shcherbatyi, et al.

Regularized empirical risk minimization with constrained labels (in contrast to fixed labels) is a remarkably general abstraction of learning. For common loss and regularization functions, this optimization problem assumes the form of a mixed integer program (MIP) whose objective function is non-convex. In this form, the problem is resistant to standard optimization techniques. We construct MIPs with the same solutions whose objective functions are convex. Specifically, we characterize the tightest convex extension of the objective function, given by the Legendre-Fenchel biconjugate. Computing values of this tightest convex extension is NP-hard. However, by applying our characterization to every function in an additive decomposition of the objective function, we obtain a class of looser convex extensions that can be computed efficiently. For some decompositions, common loss and regularization functions, we derive a closed form.


page 1

page 2

page 3

page 4


Best-first Search Algorithm for Non-convex Sparse Minimization

Non-convex sparse minimization (NSM), or ℓ_0-constrained minimization of...

On the Impossibility of Convex Inference in Human Computation

Human computation or crowdsourcing involves joint inference of the groun...

Efficient Gradient Approximation Method for Constrained Bilevel Optimization

Bilevel optimization has been developed for many machine learning tasks ...

Minimizing a Sum of Clipped Convex Functions

We consider the problem of minimizing a sum of clipped convex functions;...

Implementing regularization implicitly via approximate eigenvector computation

Regularization is a powerful technique for extracting useful information...

Reactive Failure Mitigation through Seamless Migration in Telecom Infrastructure Networks

Various methods are proposed in the literature to mitigate the failures ...

GAMA: A Novel Algorithm for Non-Convex Integer Programs

Inspired by the decomposition in the hybrid quantum-classical optimizati...