
On the convergence of gradient descent for two layer neural networks
It has been shown that gradient descent can yield the zero training loss...
read it

Representation formulas and pointwise properties for Barron functions
We study the natural function space for infinitely wide twolayer neural...
read it

A function space analysis of finite neural networks with insights from sampling theory
This work suggests using sampling theory to analyze the function space r...
read it

Barron Spaces and the Compositional Function Spaces for Neural Network Models
One of the key issues in the analysis of machine learning models is to i...
read it

CNNs are Globally Optimal Given MultiLayer Support
Stochastic Gradient Descent (SGD) is the central workhorse for training ...
read it

ReLU activated MultiLayer Neural Networks trained with Mixed Integer Linear Programs
This paper is a case study to demonstrate that, in principle, multilaye...
read it

Neural Networks are Convex Regularizers: Exact Polynomialtime Convex Optimization Formulations for TwoLayer Networks
We develop exact representations of two layer neural networks with recti...
read it
On the Banach spaces associated with multilayer ReLU networks: Function representation, approximation theory and gradient descent dynamics
We develop Banach spaces for ReLU neural networks of finite depth L and infinite width. The spaces contain all finite fully connected Llayer networks and their L^2limiting objects under bounds on the natural pathnorm. Under this norm, the unit ball in the space for Llayer networks has low Rademacher complexity and thus favorable generalization properties. Functions in these spaces can be approximated by multilayer neural networks with dimensionindependent convergence rates. The key to this work is a new way of representing functions in some form of expectations, motivated by multilayer neural networks. This representation allows us to define a new class of continuous models for machine learning. We show that the gradient flow defined this way is the natural continuous analog of the gradient descent dynamics for the associated multilayer neural networks. We show that the pathnorm increases at most polynomially under this continuous gradient flow dynamics.
READ FULL TEXT
Comments
There are no comments yet.