AI Chat AI Image Generator AI Video Text to Speech

An Essay on Optimization Mystery of Deep Learning

05/17/2019

∙

by Eugene Golikov, et al.

∙

∙

Despite the huge empirical success of deep learning, theoretical understanding of neural networks learning process is still lacking. This is the reason, why some of its features seem "mysterious". We emphasize two mysteries of deep learning: generalization mystery, and optimization mystery. In this essay we review and draw connections between several selected works concerning the latter.

page 1

page 2

page 3

page 4

research

∙ 10/30/2017

How deep learning works --The geometry of deep learning

Why and how that deep learning works well on different tasks remains a m...

0 Xiao Dong, et al. ∙

research

∙ 05/09/2021

The Modern Mathematics of Deep Learning

We describe the new field of mathematical analysis of deep learning. Thi...

0 Julius Berner, et al. ∙

research

∙ 02/22/2018

Comparative analysis of SVIT and Skype features in e-Learning process

The article discusses capabilities of the SVIT and Skype programs in ter...

0 K. S. Malakhov, et al. ∙

research

∙ 12/17/2021

Neurashed: A Phenomenological Model for Imitating Deep Learning Training

To advance deep learning methodologies in the next decade, a theoretical...

0 Weijie J. Su, et al. ∙

research

∙ 07/08/2020

Winning with Simple Learning Models: Detecting Earthquakes in Groningen, the Netherlands

Deep learning is fast emerging as a potential disruptive tool to tackle ...

0 Umair bin Waheed, et al. ∙

research

∙ 02/01/2023

A Survey of Deep Learning: From Activations to Transformers

Deep learning has made tremendous progress in the last decade. A key suc...

0 Johannes Schneider, et al. ∙

research

∙ 02/22/2020

Regression with Deep Learning for Sensor Performance Optimization

Neural networks with at least two hidden layers are called deep networks...

0 Ruthvik Vaila, et al. ∙