Deep Quantile Regression: Mitigating the Curse of Dimensionality Through Composition

07/10/2021
by   Guohao Shen, et al.
0

This paper considers the problem of nonparametric quantile regression under the assumption that the target conditional quantile function is a composition of a sequence of low-dimensional functions. We study the nonparametric quantile regression estimator using deep neural networks to approximate the target conditional quantile function. For convenience, we shall refer to such an estimator as a deep quantile regression (DQR) estimator. We show that the DQR estimator achieves the nonparametric optimal convergence rate up to a logarithmic factor determined by the intrinsic dimension of the underlying compositional structure of the conditional quantile function, not the ambient dimension of the predictor. Therefore, DQR is able to mitigate the curse of dimensionality under the assumption that the conditional quantile function has a compositional structure. To establish these results, we analyze the approximation error of a composite function by neural networks and show that the error rate only depends on the dimensions of the component functions. We apply our general results to several important statistical models often used in mitigating the curse of dimensionality, including the single index, the additive, the projection pursuit, the univariate composite, and the generalized hierarchical interaction models. We explicitly describe the prefactors in the error bounds in terms of the dimensionality of the data and show that the prefactors depends on the dimensionality linearly or quadratically in these models. We also conduct extensive numerical experiments to evaluate the effectiveness of DQR and demonstrate that it outperforms a kernel-based method for nonparametric quantile regression.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/14/2021

Deep Nonparametric Regression on Approximately Low-dimensional Manifolds

In this paper, we study the properties of nonparametric least squares re...
research
04/17/2023

Deep Neural Network Approximation of Composition Functions: with application to PINNs

In this paper, we focus on approximating a natural class of functions th...
research
06/06/2023

Estimating Generalized Additive Conditional Quantiles for Absolutely Regular Processes

We propose a nonparametric method for estimating the conditional quantil...
research
10/18/2022

Nonparametric Quantile Regression: Non-Crossing Constraints and Conformal Prediction

We propose a nonparametric quantile regression method using deep neural ...
research
02/27/2023

Nonparametric regression for repeated measurements with deep neural networks

Analysis of repeated measurements for a sample of subjects has been inte...
research
02/12/2023

Deep Neural Networks for Nonparametric Interaction Models with Diverging Dimension

Deep neural networks have achieved tremendous success due to their repre...
research
08/01/2022

How Wide Convolutional Neural Networks Learn Hierarchical Tasks

Despite their success, understanding how convolutional neural networks (...

Please sign up or login with your details

Forgot password? Click here to reset