DeepAI AI Chat
Log In Sign Up

Structured Prediction Theory Based on Factor Graph Complexity

by   Corinna Cortes, et al.
NYU college

We present a general theoretical analysis of structured prediction with a series of new results. We give new data-dependent margin guarantees for structured prediction for a very wide family of loss functions and a general family of hypotheses, with an arbitrary factor graph decomposition. These are the tightest margin bounds known for both standard multi-class and general structured prediction problems. Our guarantees are expressed in terms of a data-dependent complexity measure, factor graph complexity, which we show can be estimated from data and bounded in terms of familiar quantities. We further extend our theory by leveraging the principle of Voted Risk Minimization (VRM) and show that learning is possible even with complex factor graphs. We present new learning bounds for this advanced setting, which we use to design two new algorithms, Voted Conditional Random Field (VCRF) and Voted Structured Boosting (StructBoost). These algorithms can make use of complex features and factor graphs and yet benefit from favorable learning guarantees. We also report the results of experiments with VCRF on several datasets to validate our theory.


page 1

page 2

page 3

page 4


Relative Deviation Margin Bounds

We present a series of new and more favorable margin-based learning guar...

Minimax bounds for structured prediction

Structured prediction can be considered as a generalization of many stan...

Learning Vector-valued Functions with Local Rademacher Complexity

We consider a general family of problems of which the output space admit...

ℋ-Consistency Estimation Error of Surrogate Loss Minimizers

We present a detailed study of estimation errors in terms of surrogate l...

Structured Prediction: From Gaussian Perturbations to Linear-Time Principled Algorithms

Margin-based structured prediction commonly uses a maximum loss over all...

Learning latent variable structured prediction models with Gaussian perturbations

The standard margin-based structured prediction commonly uses a maximum ...

Accelerating Optimization via Adaptive Prediction

We present a powerful general framework for designing data-dependent opt...