DeepAI AI Chat
Log In Sign Up

Deep active inference agents using Monte-Carlo methods

06/07/2020
by   Zafeirios Fountas, et al.
University of Cambridge
UCL
15

Active inference is a Bayesian framework for understanding biological intelligence. The underlying theory brings together perception and action under one single imperative: minimizing free energy. However, despite its theoretical utility in explaining intelligence, computational implementations have been restricted to low-dimensional and idealized situations. In this paper, we present a neural architecture for building deep active inference agents operating in complex, continuous state-spaces using multiple forms of Monte-Carlo (MC) sampling. For this, we introduce a number of techniques, novel to active inference. These include: i) selecting free-energy-optimal policies via MC tree search, ii) approximating this optimal policy distribution via a feed-forward `habitual' network, iii) predicting future parameter belief updates using MC dropouts and, finally, iv) optimizing state transition precision (a high-end form of attention). Our approach enables agents to learn environmental dynamics efficiently, while maintaining task performance, in relation to reward-based counterparts. We illustrate this in a new toy environment, based on the dSprites data-set, and demonstrate that active inference agents automatically create disentangled representations that are apt for modeling state transitions. In a more complex Animal-AI environment, our agents (using the same neural architecture) are able to simulate future state transitions and actions (i.e., plan), to evince reward-directed navigation - despite temporary suspension of visual input. These results show that deep active inference - equipped with MC methods - provides a flexible framework to develop biologically-inspired intelligent agents, with applications in both machine learning and cognitive science.

READ FULL TEXT

page 6

page 14

page 15

page 16

03/02/2023

Deconstructing deep active inference

Active inference is a theory of perception, learning and decision making...
03/25/2021

Active Tree Search in Large POMDPs

Model-based planning and prospection are widely studied in both cognitiv...
07/13/2022

The Free Energy Principle for Perception and Action: A Deep Learning Perspective

The free energy principle, and its corollary active inference, constitut...
08/27/2021

Active Inference for Stochastic Control

Active inference has emerged as an alternative approach to control probl...
07/08/2019

Deep Active Inference as Variational Policy Gradients

Active Inference is a theory of action arising from neuroscience which c...
11/22/2021

Branching Time Active Inference: the theory and its generality

Over the last 10 to 15 years, active inference has helped to explain var...

Code Repositories

deep-active-inference-mc

Deep active inference agents using Monte-Carlo methods


view repo

ConativeComputing_FinalProject

None


view repo