A PAC-Bayesian Perspective on Structured Prediction with Implicit Loss Embeddings

12/07/2020
by   Théophile Cantelobre, et al.
9

Many practical machine learning tasks can be framed as Structured prediction problems, where several output variables are predicted and considered interdependent. Recent theoretical advances in structured prediction have focused on obtaining fast rates convergence guarantees, especially in the Implicit Loss Embedding (ILE) framework. PAC-Bayes has gained interest recently for its capacity of producing tight risk bounds for predictor distributions. This work proposes a novel PAC-Bayes perspective on the ILE Structured prediction framework. We present two generalization bounds, on the risk and excess risk, which yield insights into the behavior of ILE predictors. Two learning algorithms are derived from these bounds. The algorithms are implemented and their behavior analyzed, with source code available at <https://github.com/theophilec/PAC-Bayes-ILE-Structured-Prediction>.

READ FULL TEXT

page 32

page 33

page 34

page 35

research
01/16/2019

A Primer on PAC-Bayesian Learning

Generalized Bayesian learning algorithms are increasingly popular in mac...
research
08/19/2019

PAC-Bayes with Backprop

We explore a method to train probabilistic neural networks by minimizing...
research
02/12/2019

PAC-Bayes Analysis of Sentence Representation

Learning sentence vectors from an unlabeled corpus has attracted attenti...
research
06/15/2023

On Certified Generalization in Structured Prediction

In structured prediction, target objects have rich internal structure wh...
research
11/15/2021

Progress in Self-Certified Neural Networks

A learning method is self-certified if it uses all available data to sim...
research
10/10/2019

Still no free lunches: the price to pay for tighter PAC-Bayes bounds

"No free lunch" results state the impossibility of obtaining meaningful ...
research
01/15/2015

PAC-Bayes with Minimax for Confidence-Rated Transduction

We consider using an ensemble of binary classifiers for transductive pre...

Please sign up or login with your details

Forgot password? Click here to reset