AI Chat AI Image Generator AI Video Text to Speech

Representation formulas and pointwise properties for Barron functions

06/10/2020

∙

by Weinan E, et al.

∙

∙

We study the natural function space for infinitely wide two-layer neural networks and establish different representation formulae. In two cases, we describe the space explicitly up to isomorphism. Using a convenient representation, we study the pointwise properties of two-layer networks and show that functions whose singular set is fractal or curved (for example distance functions from smooth submanifolds) cannot be represented by infinitely wide two-layer networks with finite path-norm.

Weinan E
62 publications
Stephan Wojtowytsch
15 publications

page 1

page 2

page 3

page 4

research

∙ 07/30/2020

On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamics

We develop Banach spaces for ReLU neural networks of finite depth L and ...

11 Weinan E, et al. ∙

research

∙ 07/31/2020

The Kolmogorov-Arnold representation theorem revisited

There is a longstanding debate whether the Kolmogorov-Arnold representat...

0 Johannes Schmidt-Hieber, et al. ∙

research

∙ 04/15/2020

A function space analysis of finite neural networks with insights from sampling theory

This work suggests using sampling theory to analyze the function space r...

0 Raja Giryes, et al. ∙

research

∙ 03/15/2021

Representation Theorem for Matrix Product States

In this work, we investigate the universal representation capacity of th...

0 Erdong Guo, et al. ∙

research

∙ 11/20/2020

A global universality of two-layer neural networks with ReLU activations

In the present study, we investigate a universality of neural networks, ...

0 Naoya Hatano, et al. ∙

research

∙ 05/21/2020

Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels

We establish a scale separation of Kolmogorov width type between subspac...

22 Weinan E, et al. ∙

research

∙ 03/26/2021

Some properties of the parking function poset

In 1980, Edelman defined a poset on objects called the noncrossing 2-par...

0 Bérénice Delcroix-Oger, et al. ∙