Whence the Expected Free Energy?

04/17/2020
by   Beren Millidge, et al.
0

The Expected Free Energy (EFE) is a central quantity in the theory of active inference. It is the quantity that all active inference agents are mandated to minimize through action, and its decomposition into extrinsic and intrinsic value terms is key to the balance of exploration and exploitation that active inference agents evince. Despite its importance, the mathematical origins of this quantity and its relation to the Variational Free Energy (VFE) remain unclear. In this paper, we investigate the origins of the EFE in detail and show that it is not simply "the free energy in the future". We present a functional that we argue is the natural extension of the VFE, but which actively discourages exploratory behaviour, thus demonstrating that exploration does not directly follow from free energy minimization into the future. We then develop a novel objective, the Free-Energy of the Expected Future (FEEF), which possesses both the epistemic component of the EFE as well as an intuitive mathematical grounding as the divergence between predicted and desired futures.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

06/07/2020

Sophisticated Inference

Active inference offers a first principle account of sentient behaviour,...
03/11/2021

Understanding the origin of information-seeking exploration in probabilistic objectives for control

The exploration-exploitation trade-off is central to the description of ...
09/01/2021

Active Inference and Epistemic Value in Graphical Models

The Free Energy Principle (FEP) postulates that biological agents percei...
12/26/2016

Correlated signal inference by free energy exploration

The inference of correlated signal fields with unknown correlation struc...
09/24/2019

Demystifying active inference

Active inference is a first (Bayesian) principles account of how autonom...
09/21/2021

Active inference, Bayesian optimal design, and expected utility

Active inference, a corollary of the free energy principle, is a formal ...
02/28/2020

Reinforcement Learning through Active Inference

The central tenet of reinforcement learning (RL) is that agents seek to ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.