DeepAI AI Chat
Log In Sign Up

Structured Prediction: From Gaussian Perturbations to Linear-Time Principled Algorithms

by   Jean Honorio, et al.
Purdue University

Margin-based structured prediction commonly uses a maximum loss over all possible structured outputs Altun03,Collins04b,Taskar03. In natural language processing, recent work Zhang14,Zhang15 has proposed the use of the maximum loss over random structured outputs sampled independently from some proposal distribution. This method is linear-time in the number of random structured outputs and trivially parallelizable. We study this family of loss functions in the PAC-Bayes framework under Gaussian perturbations McAllester07. Under some technical conditions and up to statistical accuracy, we show that this family of loss functions produces a tighter upper bound of the Gibbs decoder distortion than commonly used methods. Thus, using the maximum loss over random structured outputs is a principled way of learning the parameter of structured prediction models. Besides explaining the experimental success of Zhang14,Zhang15, our theoretical results show that more general techniques are possible.


page 1

page 2

page 3

page 4


Learning latent variable structured prediction models with Gaussian perturbations

The standard margin-based structured prediction commonly uses a maximum ...

On the Generalization of the C-Bound to Structured Output Ensemble Methods

This paper generalizes an important result from the PAC-Bayesian literat...

A Consistent Regularization Approach for Structured Prediction

We propose and analyze a regularization approach for structured predicti...

Learning Maximum-A-Posteriori Perturbation Models for Structured Prediction in Polynomial Time

MAP perturbation models have emerged as a powerful framework for inferen...

Structured Prediction Theory Based on Factor Graph Complexity

We present a general theoretical analysis of structured prediction with ...

Learning Max-Margin Tree Predictors

Structured prediction is a powerful framework for coping with joint pred...

A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings

We propose and analyze a novel theoretical and algorithmic framework for...