Learning under Distribution Mismatch and Model Misspecification

02/10/2021
by   Mohammad Saeed Masiha, et al.
0

We study learning algorithms when there is a mismatch between the distributions of the training and test datasets of a learning algorithm. The effect of this mismatch on the generalization error and model misspecification are quantified. Moreover, we provide a connection between the generalization error and the rate-distortion theory, which allows one to utilize bounds from the rate-distortion theory to derive new bounds on the generalization error and vice versa. In particular, the rate-distortion based bound strictly improves over the earlier bound by Xu and Raginsky even when there is no mismatch. We also discuss how "auxiliary loss functions" can be utilized to obtain upper bounds on the generalization error.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

02/03/2021

Information-Theoretic Bounds on the Moments of the Generalization Error of Learning Algorithms

Generalization error bounds are critical to understanding the performanc...
11/08/2018

An Optimal Transport View on Generalization

We derive upper bounds on the generalization error of learning algorithm...
04/04/2019

Compact Error-Resilient Self-Assembly of Recursively Defined Patterns

A limitation to molecular implementations of tile-based self-assembly sy...
10/12/2018

Spherical Regression under Mismatch Corruption with Application to Automated Knowledge Translation

Motivated by a series of applications in data integration, language tran...
12/10/2021

PACMAN: PAC-style bounds accounting for the Mismatch between Accuracy and Negative log-loss

The ultimate performance of machine learning algorithms for classificati...
05/16/2020

Generalization Bounds via Information Density and Conditional Information Density

We present a general approach, based on an exponential inequality, to de...
12/19/2014

Empirically Estimable Classification Bounds Based on a New Divergence Measure

Information divergence functions play a critical role in statistics and ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.