DeepAI AI Chat
Log In Sign Up

Efficient Structured Surrogate Loss and Regularization in Structured Prediction

by   Heejin Choi, et al.

In this dissertation, we focus on several important problems in structured prediction. In structured prediction, the label has a rich intrinsic substructure, and the loss varies with respect to the predicted label and the true label pair. Structured SVM is an extension of binary SVM to adapt to such structured tasks. In the first part of the dissertation, we study the surrogate losses and its efficient methods. To minimize the empirical risk, a surrogate loss which upper bounds the loss, is used as a proxy to minimize the actual loss. Since the objective function is written in terms of the surrogate loss, the choice of the surrogate loss is important, and the performance depends on it. Another issue regarding the surrogate loss is the efficiency of the argmax label inference for the surrogate loss. Efficient inference is necessary for the optimization since it is often the most time-consuming step. We present a new class of surrogate losses named bi-criteria surrogate loss, which is a generalization of the popular surrogate losses. We first investigate an efficient method for a slack rescaling formulation as a starting point utilizing decomposability of the model. Then, we extend the algorithm to the bi-criteria surrogate loss, which is very efficient and also shows performance improvements. In the second part of the dissertation, another important issue of regularization is studied. Specifically, we investigate a problem of regularization in hierarchical classification when a structural imbalance exists in the label structure. We present a method to normalize the structure, as well as a new norm, namely shared Frobenius norm. It is suitable for hierarchical classification that adapts to the data in addition to the label structure.


page 1

page 2

page 3

page 4


Calibrated Surrogate Losses for Classification with Label-Dependent Costs

We present surrogate regret bounds for arbitrary surrogate losses in the...

On Structured Prediction Theory with Calibrated Convex Surrogate Losses

We provide novel theoretical insights on structured prediction in the co...

Leveraging Low-Rank Relations Between Surrogate Tasks in Structured Prediction

We study the interplay between surrogate methods for structured predicti...

Learning Surrogate Losses

The minimization of loss functions is the heart and soul of Machine Lear...

Minimizing Close-k Aggregate Loss Improves Classification

In classification, the de facto method for aggregating individual losses...

Towards Consistency in Adversarial Classification

In this paper, we study the problem of consistency in the context of adv...

Minimax Classification with 0-1 Loss and Performance Guarantees

Supervised classification techniques use training samples to find classi...