On Misspecification in Prediction Problems and Robustness via Improper Learning

01/13/2021
by   Annie Marsden, et al.
0

We study probabilistic prediction games when the underlying model is misspecified, investigating the consequences of predicting using an incorrect parametric model. We show that for a broad class of loss functions and parametric families of distributions, the regret of playing a "proper" predictor – one from the putative model class – relative to the best predictor in the same model class has lower bound scaling at least as √(γ n), where γ is a measure of the model misspecification to the true distribution in terms of total variation distance. In contrast, using an aggregation-based (improper) learner, one can obtain regret d log n for any underlying generating distribution, where d is the dimension of the parameter; we exhibit instances in which this is unimprovable even over the family of all learners that may play distributions in the convex hull of the parametric family. These results suggest that simple strategies for aggregating multiple learners together should be more robust, and several experiments conform to this hypothesis.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/29/2021

Sequential prediction under log-loss and misspecification

We consider the question of sequential prediction under the log-loss in ...
research
11/09/2018

Density estimation for shift-invariant multidimensional distributions

We study density estimation for classes of shift-invariant distributions...
research
11/17/2021

Fast Rates for Nonparametric Online Learning: From Realizability to Learning in Games

We study fast rates of convergence in the setting of nonparametric onlin...
research
08/16/2021

Near-Optimal No-Regret Learning in General Games

We show that Optimistic Hedge – a common variant of multiplicative-weigh...
research
11/05/2020

On the Information Complexity of Proper Learners for VC Classes in the Realizable Case

We provide a negative resolution to a conjecture of Steinke and Zakynthi...
research
12/27/2022

Further Results on the Bivariate Semi-parametric Singular Family of Distributions

General classes of bivariate distributions are well studied in literatur...
research
09/20/2009

Random scattering of bits by prediction

We investigate a population of binary mistake sequences that result from...

Please sign up or login with your details

Forgot password? Click here to reset