DeepAI AI Chat
Log In Sign Up

Tightening the Biological Constraints on Gradient-Based Predictive Coding

by   Nick Alonso, et al.
University of California, Irvine

Predictive coding (PC) is a general theory of cortical function. The local, gradient-based learning rules found in one kind of PC model have recently been shown to closely approximate backpropagation. This finding suggests that this gradient-based PC model may be useful for understanding how the brain solves the credit assignment problem. The model may also be useful for developing local learning algorithms that are compatible with neuromorphic hardware. In this paper, we modify this PC model so that it better fits biological constraints, including the constraints that neurons can only have positive firing rates and the constraint that synapses only flow in one direction. We also compute the gradient-based weight and activity updates given the modified activity values. We show that, under certain conditions, these modified PC networks perform as well or nearly as well on MNIST data as the unmodified PC model and networks trained with backpropagation.


On-chip Few-shot Learning with Surrogate Gradient Descent on a Neuromorphic Processor

Recent work suggests that synaptic plasticity dynamics in biological mod...

Minimizing Control for Credit Assignment with Strong Feedback

The success of deep learning attracted interest in whether the brain lea...

The Predictive Forward-Forward Algorithm

We propose the predictive forward-forward (PFF) algorithm for conducting...

Relaxing the Constraints on Predictive Coding Models

Predictive coding is an influential theory of cortical function which po...

Learning to solve the credit assignment problem

Backpropagation is driving today's artificial neural networks (ANNs). Ho...

Predictive Coding Approximates Backprop along Arbitrary Computation Graphs

Backpropagation of error (backprop) is a powerful algorithm for training...

Structured and Deep Similarity Matching via Structured and Deep Hebbian Networks

Synaptic plasticity is widely accepted to be the mechanism behind learni...