Characterization of the Variation Spaces Corresponding to Shallow Neural Networks

06/28/2021
by   Jonathan W. Siegel, et al.
8

We consider the variation space corresponding to a dictionary of functions in L^2(Ω) and present the basic theory of approximation in these spaces. Specifically, we compare the definition based on integral representations with the definition in terms of convex hulls. We show that in many cases, including the dictionaries corresponding to shallow ReLU^k networks and a dictionary of decaying Fourier modes, that the two definitions coincide. We also give a partial characterization of the variation space for shallow ReLU^k networks and show that the variation space with respect to the dictionary of decaying Fourier modes corresponds to the Barron spectral space.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/04/2023

Optimal rates of approximation by shallow ReLU^k neural networks and applications to nonparametric regression

We study the approximation capacity of some variation spaces correspondi...
research
07/28/2023

Weighted variation spaces and approximation by shallow ReLU networks

We investigate the approximation of functions f on a bounded domain Ω⊂ℝ^...
research
01/29/2021

Optimal Approximation Rates and Metric Entropy of ReLU^k and Cosine Networks

This article addresses several fundamental issues associated with the ap...
research
09/18/2021

Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks

We study the problem of estimating an unknown function from noisy data u...
research
02/26/2023

Bochner integrals and neural networks

A Bochner integral formula is derived that represents a function in term...
research
07/28/2023

Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networks

We study the following two related problems. The first is to determine t...
research
05/24/2019

Greedy Shallow Networks: A New Approach for Constructing and Training Neural Networks

We present a novel greedy approach to obtain a single layer neural netwo...

Please sign up or login with your details

Forgot password? Click here to reset