AI Chat AI Image Generator AI Video Text to Speech

Approximate Spectral Decomposition of Fisher Information Matrix for Simple ReLU Networks

11/30/2021

∙

by Yoshinari Takeishi, et al.

∙

∙

We investigate the Fisher information matrix (FIM) of one hidden layer networks with the ReLU activation function and obtain an approximate spectral decomposition of FIM under certain conditions. From this decomposition, we can approximate the main eigenvalues and eigenvectors. We confirmed by numerical simulation that the obtained decomposition is approximately correct when the number of hidden nodes is about 10000.

Yoshinari Takeishi
2 publications
Masazumi Iida
1 publication
Jun'ichi Takeuchi
3 publications

page 1

page 2

page 3

page 4

research

∙ 09/01/2021

Approximation Properties of Deep ReLU CNNs

This paper is devoted to establishing L^2 approximation properties for d...

11 Juncai He, et al. ∙

research

∙ 07/27/2021

Convergence of Deep ReLU Networks

We explore convergence of deep neural networks with the popular ReLU act...

0 Yuesheng Xu, et al. ∙

research

∙ 08/09/2022

On the Activation Function Dependence of the Spectral Bias of Neural Networks

Neural networks are universal function approximators which are known to ...

12 Qingguo Hong, et al. ∙

research

∙ 10/15/2019

Matrix Spectral Factorization for SA4 Multiwavelet

In this paper, we investigate Bauer's method for the matrix spectral fac...

0 Vasil Kolev, et al. ∙

research

∙ 04/19/2023

Points of non-linearity of functions generated by random neural networks

We consider functions from the real numbers to the real numbers, output ...

0 David Holmes, et al. ∙

research

∙ 07/25/2022

Decision-oriented two-parameter Fisher information sensitivity using symplectic decomposition

The eigenvalues and eigenvectors of the Fisher information matrix (FIM) ...

0 Jiannan Yang, et al. ∙

research

∙ 11/23/2022

Dual Graphs of Polyhedral Decompositions for the Detection of Adversarial Attacks

Previous work has shown that a neural network with the rectified linear ...

0 Huma Jamil, et al. ∙