Analytic Network Learning

11/20/2018
by   Kar-Ann Toh, et al.
0

Based on the property that solving the system of linear matrix equations via the column space and the row space projections boils down to an approximation in the least squares error sense, a formulation for learning the weight matrices of the multilayer network can be derived. By exploiting into the vast number of feasible solutions of these interdependent weight matrices, the learning can be performed analytically layer by layer without needing of gradient computation after an initialization. Possible initialization schemes include utilizing the data matrix as initial weights and random initialization. The study is followed by an investigation into the representation capability and the output variance of the learning scheme. An extensive experimentation on synthetic and real-world data sets validates its numerical feasibility.

READ FULL TEXT

page 22

page 23

research
10/27/2018

Gradient-Free Learning Based on the Kernel and the Range Space

In this article, we show that solving the system of linear equations by ...
research
10/22/2018

Learning from the Kernel and the Range Space

In this article, a novel approach to learning a complex function which c...
research
02/26/2021

Layer-Wise Interpretation of Deep Neural Networks Using Identity Initialization

The interpretability of neural networks (NNs) is a challenging but essen...
research
10/22/2012

Initialization of Self-Organizing Maps: Principal Components Versus Random Initialization. A Case Study

The performance of the Self-Organizing Map (SOM) algorithm is dependent ...
research
11/03/2021

The effect of synaptic weight initialization in feature-based successor representation learning

After discovering place cells, the idea of the hippocampal (HPC) functio...
research
10/28/2019

HIDRA: Head Initialization across Dynamic targets for Robust Architectures

The performance of gradient-based optimization strategies depends heavil...
research
03/05/2022

pylspack: Parallel algorithms and data structures for sketching, column subset selection, regression and leverage scores

We present parallel algorithms and data structures for three fundamental...

Please sign up or login with your details

Forgot password? Click here to reset