Network Parameter Learning Using Nonlinear Transforms, Local Representation Goals and Local Propagation Constraints

01/31/2019
by   Dimche Kostadinov, et al.
0

In this paper, we introduce a novel concept for learning of the parameters in a neural network. Our idea is grounded on modeling a learning problem that addresses a trade-off between (i) satisfying local objectives at each node and (ii) achieving desired data propagation through the network under (iii) local propagation constraints. We consider two types of nonlinear transforms which describe the network representations. One of the nonlinear transforms serves as activation function. The other one enables a locally adjusted, deviation corrective components to be included in the update of the network weights in order to enable attaining target specific representations at the last network node. Our learning principle not only provides insight into the understanding and the interpretation of the learning dynamics, but it offers theoretical guarantees over decoupled and parallel parameter estimation strategy that enables learning in synchronous and asynchronous mode. Numerical experiments validate the potential of our approach on image recognition task. The preliminary results show advantages in comparison to the state-of-the-art methods, w.r.t. the learning time and the network size while having competitive recognition accuracy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/20/2018

Network Learning with Local Propagation

This paper presents a locally decoupled network parameter learning with ...
research
01/30/2019

Clustering with Jointly Learned Nonlinear Transforms Over Discriminating Min-Max Similarity/Dissimilarity Assignment

This paper presents a novel clustering concept that is based on jointly ...
research
08/23/2023

Machine learning in parameter estimation of nonlinear systems

Accurately estimating parameters in complex nonlinear systems is crucial...
research
09/23/2020

Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation

Event-based cameras record an asynchronous stream of per-pixel brightnes...
research
11/27/2021

AIS: A nonlinear activation function for industrial safety engineering

In the task of Chinese named entity recognition based on deep learning, ...
research
07/04/2019

Neural Networks, Hypersurfaces, and Radon Transforms

Connections between integration along hypersufaces, Radon transforms, an...
research
10/13/2021

Bundle Networks: Fiber Bundles, Local Trivializations, and a Generative Approach to Exploring Many-to-one Maps

Many-to-one maps are ubiquitous in machine learning, from the image reco...

Please sign up or login with your details

Forgot password? Click here to reset