Variational Probability Flow for Biologically Plausible Training of Deep Neural Networks

11/21/2017
by   Zuozhu Liu, et al.
0

The quest for biologically plausible deep learning is driven, not just by the desire to explain experimentally-observed properties of biological neural networks, but also by the hope of discovering more efficient methods for training artificial networks. In this paper, we propose a new algorithm named Variational Probably Flow (VPF), an extension of minimum probability flow for training binary Deep Boltzmann Machines (DBMs). We show that weight updates in VPF are local, depending only on the states and firing rates of the adjacent neurons. Unlike contrastive divergence, there is no need for Gibbs confabulations; and unlike backpropagation, alternating feedforward and feedback phases are not required. Moreover, the learning algorithm is effective for training DBMs with intra-layer connections between the hidden nodes. Experiments with MNIST and Fashion MNIST demonstrate that VPF learns reasonable features quickly, reconstructs corrupted images more accurately, and generates samples with a high estimated log-likelihood. Lastly, we note that, interestingly, if an asymmetric version of VPF exists, the weight updates directly explain experimental results in Spike-Timing-Dependent Plasticity (STDP).

READ FULL TEXT

page 5

page 6

page 7

page 10

research
02/27/2019

Biologically plausible deep learning -- but how far can we go with shallow networks?

Training deep neural networks with the error backpropagation algorithm i...
research
02/23/2017

Bidirectional Backpropagation: Towards Biologically Plausible Error Signal Transmission in Neural Networks

The back-propagation (BP) algorithm has been considered the de-facto met...
research
11/24/2020

A More Biologically Plausible Local Learning Rule for ANNs

The backpropagation algorithm is often debated for its biological plausi...
research
02/24/2020

Supervised Deep Similarity Matching

We propose a novel biologically-plausible solution to the credit assignm...
research
05/22/2022

aSTDP: A More Biologically Plausible Learning

Spike-timing dependent plasticity in biological neural networks has been...
research
11/29/2019

MSTDP: A More Biologically Plausible Learning

Spike-timing dependent plasticity (STDP) which observed in the brain has...
research
10/26/2019

Making Predictive Coding Networks Generative

Predictive coding (PC) networks are a biologically interesting class of ...

Please sign up or login with your details

Forgot password? Click here to reset