Convolutional Bipartite Attractor Networks

06/08/2019
by   Michael Iuzzolino, et al.
7

In human perception and cognition, the fundamental operation that brains perform is interpretation: constructing coherent neural states from noisy, incomplete, and intrinsically ambiguous evidence. The problem of interpretation is well matched to an early and often overlooked architecture, the attractor network---a recurrent neural network that performs constraint satisfaction, imputation of missing features, and clean up of noisy data via energy minimization dynamics. We revisit attractor nets in light of modern deep learning methods, and propose a convolutional bipartite architecture with a novel training loss, activation function, and connectivity constraints. We tackle problems much larger than have been previously explored with attractor nets and demonstrate their potential for image denoising, completion, and super-resolution. We argue that this architecture is better motivated than ever-deeper feedforward models and is a viable alternative to more costly sampling-based methods on a range of supervised and unsupervised tasks.

READ FULL TEXT

page 5

page 6

page 7

page 8

page 16

page 17

research
05/22/2018

State-Denoised Recurrent Neural Networks

Recurrent neural networks (RNNs) are difficult to train on sequence proc...
research
09/10/2016

Rectifier Neural Network with a Dual-Pathway Architecture for Image Denoising

Recently deep neural networks based on tanh activation function have sho...
research
09/17/2017

Neural Affine Grayscale Image Denoising

We propose a new grayscale image denoiser, dubbed as Neural Affine Image...
research
04/28/2019

An approach to image denoising using manifold approximation without clean images

Image restoration has been an extensively researched topic in numerous f...
research
09/18/2019

RUN-CSP: Unsupervised Learning of Message Passing Networks for Binary Constraint Satisfaction Problems

Constraint satisfaction problems form an important and wide class of com...

Please sign up or login with your details

Forgot password? Click here to reset