Strong overall error analysis for the training of artificial neural networks via random initializations

12/15/2020
by   Arnulf Jentzen, et al.
0

Although deep learning based approximation algorithms have been applied very successfully to numerous problems, at the moment the reasons for their performance are not entirely understood from a mathematical point of view. Recently, estimates for the convergence of the overall error have been obtained in the situation of deep supervised learning, but with an extremely slow rate of convergence. In this note we partially improve on these estimates. More specifically, we show that the depth of the neural network only needs to increase much slower in order to obtain the same rate of approximation. The results hold in the case of an arbitrary stochastic optimization algorithm with i.i.d. random initializations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/03/2020

Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation

In spite of the accomplishments of deep learning based algorithms in num...
research
09/30/2019

Full error analysis for the training of deep neural networks

Deep learning algorithms have been applied very successfully in recent y...
research
08/18/2023

Baird Counterexample Is Solved: with an example of How to Debug a Two-time-scale Algorithm

Baird counterexample was proposed by Leemon Baird in 1995, first used to...
research
08/11/2019

Space-time error estimates for deep neural network approximations for differential equations

Over the last few years deep artificial neural networks (DNNs) have very...
research
06/23/2023

A new approach to generalisation error of machine learning algorithms: Estimates and convergence

In this work we consider a model problem of deep neural learning, namely...
research
12/10/2020

Asymptotic study of stochastic adaptive algorithm in non-convex landscape

This paper studies some asymptotic properties of adaptive algorithms wid...
research
10/26/2017

Maximum Principle Based Algorithms for Deep Learning

The continuous dynamical system approach to deep learning is explored in...

Please sign up or login with your details

Forgot password? Click here to reset