DeepAI
Log In Sign Up

Topological Gradient-based Competitive Learning

08/21/2020
by   Pietro Barbiero, et al.
0

Topological learning is a wide research area aiming at uncovering the mutual spatial relationships between the elements of a set. Some of the most common and oldest approaches involve the use of unsupervised competitive neural networks. However, these methods are not based on gradient optimization which has been proven to provide striking results in feature extraction also in unsupervised learning. Unfortunately, by focusing mostly on algorithmic efficiency and accuracy, deep clustering techniques are composed of overly complex feature extractors, while using trivial algorithms in their top layer. The aim of this work is to present a novel comprehensive theory aspiring at bridging competitive learning with gradient-based learning, thus allowing the use of extremely powerful deep neural networks for feature extraction and projection combined with the remarkable flexibility and expressiveness of competitive learning. In this paper we fully demonstrate the theoretical equivalence of two novel gradient-based competitive layers. Preliminary experiments show how the dual approach, trained on the transpose of the input matrix i.e. X^T, lead to faster convergence rate and higher training accuracy both in low and high-dimensional scenarios.

READ FULL TEXT
09/06/2020

Gradient-based Competitive Learning: Theory

Deep learning has been widely used for supervised learning and classific...
09/04/2020

Improving Self-Organizing Maps with Unsupervised Feature Extraction

The Self-Organizing Map (SOM) is a brain-inspired neural model that is v...
07/23/2021

Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks

With the rise of deep neural networks, the challenge of explaining the p...
05/06/2022

Beyond backpropagation: implicit gradients for bilevel optimization

This paper reviews gradient-based techniques to solve bilevel optimizati...
06/24/2012

Practical recommendations for gradient-based training of deep architectures

Learning algorithms related to artificial neural networks and in particu...
10/27/2021

CBIR using Pre-Trained Neural Networks

Much of the recent research work in image retrieval, has been focused ar...
03/02/2021

Categorical Foundations of Gradient-Based Learning

We propose a categorical foundation of gradient-based machine learning a...