The Neural Coding Framework for Learning Generative Models

12/07/2020
by   Alexander Ororbia, et al.
0

Neural generative models can be used to learn complex probability distributions from data, to sample from them, and to produce probability density estimates. We propose a novel neural generative model inspired by the theory of predictive processing in the brain. According to predictive processing theory, the neurons in the brain form a hierarchy in which neurons in one level form expectations about sensory inputs from another level. These neurons update their local models based on differences between their expectations and the observed signals. In a similar way, artificial neurons in our generative model predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality. This neural generative model performs very well in practice. On a variety of benchmark datasets and metrics, it either remains competitive with or significantly outperforms other generative models with similar functionality (such as the variational auto-encoder).

READ FULL TEXT

page 7

page 8

research
04/29/2019

Generative models as parsimonious descriptions of sensorimotor loops

The Bayesian brain hypothesis, predictive processing and variational fre...
research
07/14/2023

Brain in the Dark: Design Principles for Neuro-mimetic Learning and Inference

Even though the brain operates in pure darkness, within the skull, it ca...
research
12/21/2020

Building LEGO Using Deep Generative Models of Graphs

Generative models are now used to create a variety of high-quality digit...
research
08/23/2023

Improving Generative Model-based Unfolding with Schrödinger Bridges

Machine learning-based unfolding has enabled unbinned and high-dimension...
research
11/22/2022

Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images

In this work, we develop convolutional neural generative coding (Conv-NG...
research
01/28/2019

Lie Group Auto-Encoder

In this paper, we propose an auto-encoder based generative neural networ...
research
12/15/2020

Bayes Meets Entailment and Prediction: Commonsense Reasoning with Non-monotonicity, Paraconsistency and Predictive Accuracy

The recent success of Bayesian methods in neuroscience and artificial in...

Please sign up or login with your details

Forgot password? Click here to reset