Active Predictive Coding: A Unified Neural Framework for Learning Hierarchical World Models for Perception and Planning

by   Rajesh P. N. Rao, et al.

Predictive coding has emerged as a prominent model of how the brain learns through predictions, anticipating the importance accorded to predictive learning in recent AI architectures such as transformers. Here we propose a new framework for predictive coding called active predictive coding which can learn hierarchical world models and solve two radically different open problems in AI: (1) how do we learn compositional representations, e.g., part-whole hierarchies, for equivariant vision? and (2) how do we solve large-scale planning problems, which are hard for traditional reinforcement learning, by composing complex action sequences from primitive policies? Our approach exploits hypernetworks, self-supervised learning and reinforcement learning to learn hierarchical world models that combine task-invariant state transition networks and task-dependent policy networks at multiple abstraction levels. We demonstrate the viability of our approach on a variety of vision datasets (MNIST, FashionMNIST, Omniglot) as well as on a scalable hierarchical planning problem. Our results represent, to our knowledge, the first demonstration of a unified solution to the part-whole learning problem posed by Hinton, the nested reference frames problem posed by Hawkins, and the integrated state-action hierarchy learning problem in reinforcement learning.


page 5

page 6

page 7

page 8

page 14


Active Predictive Coding Networks: A Neural Solution to the Problem of Learning Reference Frames and Part-Whole Hierarchies

We introduce Active Predictive Coding Networks (APCNs), a new class of n...

General Intelligence Requires Rethinking Exploration

We are at the cusp of a transition from "learning from data" to "learnin...

Self-Calibrating Active Binocular Vision via Active Efficient Coding with Deep Autoencoders

We present a model of the self-calibration of active binocular vision co...

Differentiable Generalised Predictive Coding

This paper deals with differentiable dynamical models congruent with neu...

Reinforcement Learning in System Identification

System identification, also known as learning forward models, transfer f...

Learning Compositional Neural Programs for Continuous Control

We propose a novel solution to challenging sparse-reward, continuous con...

Consciousness is learning: predictive processing systems that learn by binding may perceive themselves as conscious

Machine learning algorithms have achieved superhuman performance in spec...

Please sign up or login with your details

Forgot password? Click here to reset