DeepAI AI Chat
Log In Sign Up

Deep Nonparametric Regression on Approximately Low-dimensional Manifolds

by   Yuling Jiao, et al.

In this paper, we study the properties of nonparametric least squares regression using deep neural networks. We derive non-asymptotic upper bounds for the prediction error of the empirical risk minimizer for feedforward deep neural regression. Our error bounds achieve the minimax optimal rate and significantly improve over the existing ones in the sense that they depend linearly or quadratically on the dimension d of the predictor, instead of exponentially on d. We show that the neural regression estimator can circumvent the curse of dimensionality under the assumption that the predictor is supported on an approximate low-dimensional manifold. This assumption differs from the structural condition imposed on the target regression function and is weaker and more realistic than the exact low-dimensional manifold support assumption in the existing literature. We investigate how the prediction error of the neural regression estimator depends on the structure of neural networks and propose a notion of network relative efficiency between two types of neural networks, which provides a quantitative measure for evaluating the relative merits of different network structures. Our results are derived under weaker assumptions on the data distribution, the target regression function and the neural network structure than those in the existing literature.


page 1

page 2

page 3

page 4


Robust Nonparametric Regression with Deep Neural Networks

In this paper, we study the properties of robust nonparametric estimatio...

Deep Quantile Regression: Mitigating the Curse of Dimensionality Through Composition

This paper considers the problem of nonparametric quantile regression un...

Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces

Learning operators between infinitely dimensional spaces is an important...

Deep Learning and MARS: A Connection

We consider least squares regression estimates using deep neural network...

Relative Transfer Function Inverse Regression from Low Dimensional Manifold

In room acoustic environments, the Relative Transfer Functions (RTFs) ar...

On Deep Instrumental Variables Estimate

The endogeneity issue is fundamentally important as many empirical appli...

A Theoretical Analysis of Deep Neural Networks for Texture Classification

We investigate the use of Deep Neural Networks for the classification of...