Understanding over-parameterized deep networks by geometrization

02/11/2019
by   Xiao Dong, et al.
0

A complete understanding of the widely used over-parameterized deep networks is a key step for AI. In this work we try to give a geometric picture of over-parameterized deep networks using our geometrization scheme. We show that the Riemannian geometry of network complexity plays a key role in understanding the basic properties of over-parameterizaed deep networks, including the generalization, convergence and parameter sensitivity. We also point out deep networks share lots of similarities with quantum computation systems. This can be regarded as a strong support of our proposal that geometrization is not only the bible for physics, it is also the key idea to understand deep learning systems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/12/2019

Deep network as memory space: complexity, generalization, disentangled representation and interpretability

By bridging deep networks and physics, the programme of geometrization o...
research
01/06/2019

Geometrization of deep networks for the interpretability of deep learning systems

How to understand deep learning systems remains an open problem. In this...
research
10/30/2017

How deep learning works --The geometry of deep learning

Why and how that deep learning works well on different tasks remains a m...
research
11/16/2019

Exploiting Oxide Based Resistive RAM Variability for Probabilistic AI Hardware Design

Uncertainty plays a key role in real-time machine learning. As a signifi...
research
07/11/2019

A General Decoupled Learning Framework for Parameterized Image Operators

Many different deep networks have been used to approximate, accelerate o...
research
04/29/2021

A neural anisotropic view of underspecification in deep learning

The underspecification of most machine learning pipelines means that we ...
research
03/19/2022

Deep Learning Generalization, Extrapolation, and Over-parameterization

We study the generalization of over-parameterized deep networks (for ima...

Please sign up or login with your details

Forgot password? Click here to reset