Learning Surrogate Losses

05/24/2019
by   Josif Grabocka, et al.
0

The minimization of loss functions is the heart and soul of Machine Learning. In this paper, we propose an off-the-shelf optimization approach that can minimize virtually any non-differentiable and non-decomposable loss function (e.g. Miss-classification Rate, AUC, F1, Jaccard Index, Mathew Correlation Coefficient, etc.) seamlessly. Our strategy learns smooth relaxation versions of the true losses by approximating them through a surrogate neural network. The proposed loss networks are set-wise models which are invariant to the order of mini-batch instances. Ultimately, the surrogate losses are learned jointly with the prediction model via bilevel optimization. Empirical results on multiple datasets with diverse real-life loss functions compared with state-of-the-art baselines demonstrate the efficiency of learning surrogate losses.

READ FULL TEXT
research
12/24/2015

The Lovász Hinge: A Novel Convex Surrogate for Submodular Losses

Learning with non-modular losses is an important problem when sets of pr...
research
02/27/2021

Learning with Smooth Hinge Losses

Due to the non-smoothness of the Hinge loss in SVM, it is difficult to o...
research
02/26/2022

Relational Surrogate Loss Learning

Evaluation metrics in machine learning are often hardly taken as loss fu...
research
08/24/2021

sigmoidF1: A Smooth F1 Score Surrogate Loss for Multilabel Classification

Multiclass multilabel classification refers to the task of attributing m...
research
10/15/2020

Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation

We propose a general framework for searching surrogate losses for mainst...
research
09/14/2018

Efficient Structured Surrogate Loss and Regularization in Structured Prediction

In this dissertation, we focus on several important problems in structur...
research
03/10/2018

A Minimax Surrogate Loss Approach to Conditional Difference Estimation

We present a new machine learning approach to estimate personalized trea...

Please sign up or login with your details

Forgot password? Click here to reset