The Variability of Model Specification

10/06/2021
by   Joseph R. Barr, et al.
9

It's regarded as an axiom that a good model is one that compromises between bias and variance. The bias is measured in training cost, while the variance of a (say, regression) model is measure by the cost associated with a validation set. If reducing bias is the goal, one will strive to fetch as complex a model as necessary, but complexity is invariably coupled with variance: greater complexity implies greater variance. In practice, driving training cost to near zero does not pose a fundamental problem; in fact, a sufficiently complex decision tree is perfectly capable of driving training cost to zero; however, the problem is often with controlling the model's variance. We investigate various regression model frameworks, including generalized linear models, Cox proportional hazard models, ARMA, and illustrate how misspecifying a model affects the variance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/10/2022

Bias-variance decomposition of overparameterized regression with random linear features

In classical statistics, the bias-variance trade-off describes how varyi...
research
10/26/2020

Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models

The bias-variance trade-off is a central concept in supervised learning....
research
03/17/2021

Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition

Adversarially trained models exhibit a large generalization gap: they ca...
research
09/24/2011

Bias Plus Variance Decomposition for Survival Analysis Problems

Bias - variance decomposition of the expected error defined for regressi...
research
10/19/2018

A Modern Take on the Bias-Variance Tradeoff in Neural Networks

We revisit the bias-variance tradeoff for neural networks in light of mo...
research
11/18/2020

Bias-Variance Trade-off and Overlearning in Dynamic Decision Problems

Modern Monte Carlo-type approaches to dynamic decision problems face the...
research
09/09/2019

Bias-Variance Games

Firms engaged in electronic commerce increasingly rely on machine learni...

Please sign up or login with your details

Forgot password? Click here to reset