A Convex Surrogate Operator for General Non-Modular Loss Functions

04/12/2016
by   Jiaqian Yu, et al.
0

Empirical risk minimization frequently employs convex surrogates to underlying discrete loss functions in order to achieve computational tractability during optimization. However, classical convex surrogates can only tightly bound modular loss functions, sub-modular functions or supermodular functions separately while maintaining polynomial time computation. In this work, a novel generic convex surrogate for general non-modular loss functions is introduced, which provides for the first time a tractable solution for loss functions that are neither super-modular nor submodular. This convex surro-gate is based on a submodular-supermodular decomposition for which the existence and uniqueness is proven in this paper. It takes the sum of two convex surrogates that separately bound the supermodular component and the submodular component using slack-rescaling and the Lovász hinge, respectively. It is further proven that this surrogate is convex , piecewise linear, an extension of the loss function, and for which subgradient computation is polynomial time. Empirical results are reported on a non-submodular loss based on the Sørensen-Dice difference function, and a real-world face track dataset with tens of thousands of frames, demonstrating the improved performance, efficiency, and scalabil-ity of the novel convex surrogate.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/24/2015

The Lovász Hinge: A Novel Convex Surrogate for Submodular Losses

Learning with non-modular losses is an important problem when sets of pr...
research
02/18/2022

Signal Decomposition Using Masked Proximal Operators

We consider the well-studied problem of decomposing a vector time series...
research
06/27/2020

Concave Aspects of Submodular Functions

Submodular Functions are a special class of set functions, which general...
research
09/09/2015

Sensor Selection by Linear Programming

We learn sensor trees from training data to minimize sensor acquisition ...
research
12/17/2018

Differentially Private Empirical Risk Minimization in Non-interactive Local Model via Polynomial of Inner Product Approximation

In this paper, we study the Empirical Risk Minimization problem in the n...
research
03/16/2022

The Structured Abstain Problem and the Lovász Hinge

The Lovász hinge is a convex surrogate recently proposed for structured ...
research
11/11/2020

Empirical Risk Minimization in the Non-interactive Local Model of Differential Privacy

In this paper, we study the Empirical Risk Minimization (ERM) problem in...

Please sign up or login with your details

Forgot password? Click here to reset