DeepAI AI Chat
Log In Sign Up

Robust Nonparametric Regression with Deep Neural Networks

by   Guohao Shen, et al.
The University of Iowa
Wuhan University
The Chinese University of Hong Kong

In this paper, we study the properties of robust nonparametric estimation using deep neural networks for regression models with heavy tailed error distributions. We establish the non-asymptotic error bounds for a class of robust nonparametric regression estimators using deep neural networks with ReLU activation under suitable smoothness conditions on the regression function and mild conditions on the error term. In particular, we only assume that the error distribution has a finite p-th moment with p greater than one. We also show that the deep robust regression estimators are able to circumvent the curse of dimensionality when the distribution of the predictor is supported on an approximate lower-dimensional set. An important feature of our error bound is that, for ReLU neural networks with network width and network size (number of parameters) no more than the order of the square of the dimensionality d of the predictor, our excess risk bounds depend sub-linearly on d. Our assumption relaxes the exact manifold support assumption, which could be restrictive and unrealistic in practice. We also relax several crucial assumptions on the data distribution, the target regression function and the neural networks required in the recent literature. Our simulation studies demonstrate that the robust methods can significantly outperform the least squares method when the errors have heavy-tailed distributions and illustrate that the choice of loss function is important in the context of deep nonparametric regression.


page 1

page 2

page 3

page 4


Deep Nonparametric Regression on Approximately Low-dimensional Manifolds

In this paper, we study the properties of nonparametric least squares re...

How do noise tails impact on deep ReLU networks?

This paper investigates the stability of deep ReLU neural networks for n...

On Least Squares Estimation under Heteroscedastic and Heavy-Tailed Errors

We consider least squares estimation in a general nonparametric regressi...

Deep Learning and MARS: A Connection

We consider least squares regression estimates using deep neural network...

Non-Asymptotic Guarantees for Robust Statistical Learning under (1+ε)-th Moment Assumption

There has been a surge of interest in developing robust estimators for m...

Measurement error models: from nonparametric methods to deep neural networks

The success of deep learning has inspired recent interests in applying n...

Nonparametric regression for repeated measurements with deep neural networks

Analysis of repeated measurements for a sample of subjects has been inte...