DeepAI AI Chat
Log In Sign Up

Deep active inference agents using Monte-Carlo methods

by   Zafeirios Fountas, et al.
University of Cambridge

Active inference is a Bayesian framework for understanding biological intelligence. The underlying theory brings together perception and action under one single imperative: minimizing free energy. However, despite its theoretical utility in explaining intelligence, computational implementations have been restricted to low-dimensional and idealized situations. In this paper, we present a neural architecture for building deep active inference agents operating in complex, continuous state-spaces using multiple forms of Monte-Carlo (MC) sampling. For this, we introduce a number of techniques, novel to active inference. These include: i) selecting free-energy-optimal policies via MC tree search, ii) approximating this optimal policy distribution via a feed-forward `habitual' network, iii) predicting future parameter belief updates using MC dropouts and, finally, iv) optimizing state transition precision (a high-end form of attention). Our approach enables agents to learn environmental dynamics efficiently, while maintaining task performance, in relation to reward-based counterparts. We illustrate this in a new toy environment, based on the dSprites data-set, and demonstrate that active inference agents automatically create disentangled representations that are apt for modeling state transitions. In a more complex Animal-AI environment, our agents (using the same neural architecture) are able to simulate future state transitions and actions (i.e., plan), to evince reward-directed navigation - despite temporary suspension of visual input. These results show that deep active inference - equipped with MC methods - provides a flexible framework to develop biologically-inspired intelligent agents, with applications in both machine learning and cognitive science.


page 6

page 14

page 15

page 16


Deconstructing deep active inference

Active inference is a theory of perception, learning and decision making...

Active Tree Search in Large POMDPs

Model-based planning and prospection are widely studied in both cognitiv...

The Free Energy Principle for Perception and Action: A Deep Learning Perspective

The free energy principle, and its corollary active inference, constitut...

Active Inference for Stochastic Control

Active inference has emerged as an alternative approach to control probl...

Deep Active Inference as Variational Policy Gradients

Active Inference is a theory of action arising from neuroscience which c...

Branching Time Active Inference: the theory and its generality

Over the last 10 to 15 years, active inference has helped to explain var...

Code Repositories


Deep active inference agents using Monte-Carlo methods

view repo



view repo