Deep neural network approximation theory for high-dimensional functions

The purpose of this article is to develop machinery to study the capacity of deep neural networks (DNNs) to approximate high-dimensional functions. In particular, we show that DNNs have the expressive power to overcome the curse of dimensionality in the approximation of a large class of functions. More precisely, we prove that these functions can be approximated by DNNs on compact sets such that the number of parameters necessary to represent the approximating DNNs grows at most polynomially in the reciprocal 1/ε of the approximation accuracy ε>0 and in the input dimension d∈ℕ ={1,2,3,…}. To this end, we introduce certain approximation spaces, consisting of sequences of functions that can be efficiently approximated by DNNs. We then establish closure properties which we combine with known and new bounds on the number of parameters necessary to approximate locally Lipschitz continuous functions, maximum functions, and product functions by DNNs. The main result of this article demonstrates that DNNs have sufficient expressiveness to approximate certain sequences of functions which can be constructed by means of a finite number of compositions using locally Lipschitz continuous functions, maxima, and products without the curse of dimensionality.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/12/2023

Deep neural network approximation of composite functions without the curse of dimensionality

In this article we identify a general class of high-dimensional continuo...
research
12/09/2019

Efficient approximation of high-dimensional functions with deep neural networks

In this paper, we develop an approximation theory for deep neural networ...
research
05/17/2021

Universal Regular Conditional Distributions

We introduce a general framework for approximating regular conditional d...
research
02/13/2022

Local approximation of operators

Many applications, such as system identification, classification of time...
research
08/28/2019

Deep neural network approximations for Monte Carlo algorithms

Recently, it has been proposed in the literature to employ deep neural n...
research
06/22/2018

On the Spectral Bias of Deep Neural Networks

It is well known that over-parametrized deep neural networks (DNNs) are ...

Please sign up or login with your details

Forgot password? Click here to reset