Almost Surely Asymptotic Freeness for Jacobian Spectrum of Deep Network

08/11/2019
by   Tomohiro Hayase, et al.
0

Free probability theory helps us to understand Jacobian spectrum of deep neural networks. We rigorously show almost surely asymptotic freeness of layer-wise Jacobians.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/30/2018

The asymptotic spectrum of graphs and the Shannon capacity

We introduce the asymptotic spectrum of graphs and apply the theory of a...
research
07/31/2018

Spectrum concentration in deep residual learning: a free probability appproach

We revisit the initialization of deep residual networks (ResNets) by int...
research
08/04/2023

Deep neural networks from the perspective of ergodic theory

The design of deep neural networks remains somewhat of an art rather tha...
research
03/24/2021

Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case

Free Probability Theory (FPT) provides rich knowledge for handling mathe...
research
04/17/2019

DeepNovoV2: Better de novo peptide sequencing with deep learning

We introduce DeepNovoV2, the state-of-the-art neural networks based mode...
research
06/24/2019

A Review on Neural Network Models of Schizophrenia and Autism Spectrum Disorder

This survey presents the most relevant neural network models of autism s...
research
03/22/2021

Thomson's Multitaper Method Revisited

Thomson's multitaper method estimates the power spectrum of a signal fro...

Please sign up or login with your details

Forgot password? Click here to reset