Statistical Optimality of Interpolated Nearest Neighbor Algorithms

10/05/2018
by   Yue Xing, et al.
0

In the era of deep learning, understanding over-fitting phenomenon becomes increasingly important. It is observed that carefully designed deep neural networks achieve small testing error even the training error is close to zero. One possible explanation is that for many modern machine learning algorithms, over-fitting can greatly reduces the estimation bias, while not increases the estimation variance too much. To illustrate the above idea, we prove that our interpolated nearest neighbors algorithm achieves the minimax optimal rate in both regression and classification regimes, and observe that they are empirically better than the traditional k nearest neighbor method in some cases.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/08/2020

A Nearest Neighbor Characterization of Lebesgue Points in Metric Measure Spaces

The property of almost every point being a Lebesgue point has proven to ...
research
09/25/2019

Benefit of Interpolation in Nearest Neighbor Algorithms

The over-parameterized models attract much attention in the era of data ...
research
02/05/2022

One-Nearest-Neighbor Search is All You Need for Minimax Optimal Regression and Classification

Recently, Qiao, Duan, and Cheng (2019) proposed a distributed nearest-ne...
research
11/22/2017

Causal nearest neighbor rules for optimal treatment regimes

The estimation of optimal treatment regimes is of considerable interest ...
research
04/20/2022

One-Class Model for Fabric Defect Detection

An automated and accurate fabric defect inspection system is in high dem...
research
06/13/2018

Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate

Many modern machine learning models are trained to achieve zero or near-...
research
06/02/2019

On Testing for Parameters in Ising Models

We consider testing for the parameters of Ferromagnetic Ising models. Wh...

Please sign up or login with your details

Forgot password? Click here to reset