Sophisticated Inference

06/07/2020
by   Karl Friston, et al.
27

Active inference offers a first principle account of sentient behaviour, from which special and important cases can be derived, e.g., reinforcement learning, active learning, Bayes optimal inference, Bayes optimal design, etc. Active inference resolves the exploitation-exploration dilemma in relation to prior preferences, by placing information gain on the same footing as reward or value. In brief, active inference replaces value functions with functionals of (Bayesian) beliefs, in the form of an expected (variational) free energy. In this paper, we consider a sophisticated kind of active inference, using a recursive form of expected free energy. Sophistication describes the degree to which an agent has beliefs about beliefs. We consider agents with beliefs about the counterfactual consequences of action for states of affairs and beliefs about those latent states. In other words, we move from simply considering beliefs about 'what would happen if I did that' to 'what would I believe about what would happen if I did that'. The recursive form of the free energy functional effectively implements a deep tree search over actions and outcomes in the future. Crucially, this search is over sequences of belief states, as opposed to states per se. We illustrate the competence of this scheme, using numerical simulations of deep decision problems.

READ FULL TEXT

Authors

page 13

page 23

page 25

page 27

page 29

page 30

09/17/2020

The relationship between dynamic programming and active inference: the discrete, finite-horizon case

Active inference is a normative framework for generating behaviour based...
09/24/2019

Demystifying active inference

Active inference is a first (Bayesian) principles account of how autonom...
04/17/2020

Whence the Expected Free Energy?

The Expected Free Energy (EFE) is a central quantity in the theory of ac...
09/21/2021

Active inference, Bayesian optimal design, and expected utility

Active inference, a corollary of the free energy principle, is a formal ...
12/27/2021

Interpreting Dynamical Systems as Bayesian Reasoners

A central concept in active inference is that the internal states of a p...
07/18/2020

Modulation of viability signals for self-regulatory control

We revisit the role of instrumental value as a driver of adaptive behavi...
09/01/2021

Active Inference and Epistemic Value in Graphical Models

The Free Energy Principle (FEP) postulates that biological agents percei...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.