Neural networks with redundant representation: detecting the undetectable

11/28/2019
by   Elena Agliari, et al.
0

We consider a three-layer Sejnowski machine and show that features learnt via contrastive divergence have a dual representation as patterns in a dense associative memory of order P=4. The latter is known to be able to Hebbian-store an amount of patterns scaling as N^P-1, where N denotes the number of constituting binary neurons interacting P-wisely. We also prove that, by keeping the dense associative network far from the saturation regime (namely, allowing for a number of patterns scaling only linearly with N, while P>2) such a system is able to perform pattern recognition far below the standard signal-to-noise threshold. In particular, a network with P=4 is able to retrieve information whose intensity is O(1) even in the presence of a noise O(√(N)) in the large N limit. This striking skill stems from a redundancy representation of patterns – which is afforded given the (relatively) low-load information storage – and it contributes to explain the impressive abilities in pattern recognition exhibited by new-generation neural networks. The whole theory is developed rigorously, at the replica symmetric level of approximation, and corroborated by signal-to-noise analysis and Monte Carlo simulations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/08/2023

Parallel Learning by Multitasking Neural Networks

A modern challenge of Artificial Intelligence is learning multiple patte...
research
12/02/2019

Interpolating between boolean and extremely high noisy patterns through Minimal Dense Associative Memories

Recently, Hopfield and Krotov introduced the concept of dense associati...
research
11/25/2022

Dense Hebbian neural networks: a replica symmetric picture of supervised learning

We consider dense, associative neural-networks trained by a teacher (i.e...
research
10/29/2018

Dreaming neural networks: forgetting spurious memories and reinforcing pure ones

The standard Hopfield model for associative neural networks accounts for...
research
07/17/2023

Statistical Mechanics of Learning via Reverberation in Bidirectional Associative Memories

We study bi-directional associative neural networks that, exposed to noi...
research
09/08/2005

Achievable Rates for Pattern Recognition

Biological and machine pattern recognition systems face a common challen...
research
11/17/2022

Thermodynamics of bidirectional associative memories

In this paper we investigate the equilibrium properties of bidirectional...

Please sign up or login with your details

Forgot password? Click here to reset