Neural network approximation and estimation of classifiers with classification boundary in a Barron class

11/18/2020

∙

We prove bounds for the approximation and estimation of certain classification functions using ReLU neural networks. Our estimation bounds provide a priori performance guarantees for empirical risk minimization using networks of a suitable size, depending on the number of training samples available. The obtained approximation and estimation rates are independent of the dimension of the input, showing that the curse of dimension can be overcome in this setting; in fact, the input dimension only enters in the form of a polynomial factor. Regarding the regularity of the target classification function, we assume the interfaces between the different classes to be locally of Barron-type. We complement our results by studying the relations between various Barron-type spaces that have been proposed in the literature. These spaces differ substantially more from each other than the current literature suggests.

READ FULL TEXT

Neural network approximation and estimation of classifiers with classification boundary in a Barron class

Optimal learning of high-dimensional classification problems using deep neural networks

Approximation in shift-invariant spaces with deep ReLU neural networks

Tractability of approximation by general shallow networks

Sobolev-type embeddings for neural network approximation spaces

Training ReLU networks to high uniform accuracy is intractable

Learning with tree tensor networks: complexity estimates and model selection

Sparse approximation of triangular transports on bounded domains

Neural network approximation and estimation of classifiers with classification boundary in a Barron class

Related Research

Optimal learning of high-dimensional classification problems using deep neural networks

Approximation in shift-invariant spaces with deep ReLU neural networks

Tractability of approximation by general shallow networks

Sobolev-type embeddings for neural network approximation spaces

Training ReLU networks to high uniform accuracy is intractable

Learning with tree tensor networks: complexity estimates and model selection

Sparse approximation of triangular transports on bounded domains