Gradient-based Competitive Learning: Theory

09/06/2020
by   Giansalvo Cirrincione, et al.
0

Deep learning has been widely used for supervised learning and classification/regression problems. Recently, a novel area of research has applied this paradigm to unsupervised tasks; indeed, a gradient-based approach extracts, efficiently and autonomously, the relevant features for handling input data. However, state-of-the-art techniques focus mostly on algorithmic efficiency and accuracy rather than mimic the input manifold. On the contrary, competitive learning is a powerful tool for replicating the input distribution topology. This paper introduces a novel perspective in this area by combining these two techniques: unsupervised gradient-based and competitive learning. The theory is based on the intuition that neural networks are able to learn topological structures by working directly on the transpose of the input matrix. At this purpose, the vanilla competitive layer and its dual are presented. The former is just an adaptation of a standard competitive layer for deep clustering, while the latter is trained on the transposed matrix. Their equivalence is extensively proven both theoretically and experimentally. However, the dual layer is better suited for handling very high-dimensional datasets. The proposed approach has a great potential as it can be generalized to a vast selection of topological learning tasks, such as non-stationary and hierarchical clustering; furthermore, it can also be integrated within more complex architectures such as autoencoders and generative adversarial networks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/21/2020

Topological Gradient-based Competitive Learning

Topological learning is a wide research area aiming at uncovering the mu...
research
11/24/2021

LDDMM meets GANs: Generative Adversarial Networks for diffeomorphic registration

The purpose of this work is to contribute to the state of the art of dee...
research
05/29/2019

A Topology Layer for Machine Learning

Topology applied to real world data using persistent homology has starte...
research
07/16/2018

Meta-Learning with Latent Embedding Optimization

Gradient-based meta-learning techniques are both widely applicable and p...
research
03/03/2022

Curvature Graph Generative Adversarial Networks

Generative adversarial network (GAN) is widely used for generalized and ...
research
05/07/2021

GANTL: Towards Practical and Real-Time Topology Optimization with Conditional GANs and Transfer Learning

Many machine learning methods have been recently developed to circumvent...
research
07/19/2018

The Deep Kernelized Autoencoder

Autoencoders learn data representations (codes) in such a way that the i...

Please sign up or login with your details

Forgot password? Click here to reset