Active Inference and Epistemic Value in Graphical Models

09/01/2021
by   Thijs van de Laar, et al.
0

The Free Energy Principle (FEP) postulates that biological agents perceive and interact with their environment in order to minimize a Variational Free Energy (VFE) with respect to a generative model of their environment. The inference of a policy (future control sequence) according to the FEP is known as Active Inference (AIF). The AIF literature describes multiple VFE objectives for policy planning that lead to epistemic (information-seeking) behavior. However, most objectives have limited modeling flexibility. This paper approaches epistemic behavior from a constrained Bethe Free Energy (CBFE) perspective. Crucially, variational optimization of the CBFE can be expressed in terms of message passing on free-form generative models. The key intuition behind the CBFE is that we impose a point-mass constraint on predicted outcomes, which explicitly encodes the assumption that the agent will make observations in the future. We interpret the CBFE objective in terms of its constituent behavioral drives. We then illustrate resulting behavior of the CBFE by planning and interacting with a simulated T-maze environment. Simulations for the T-maze task illustrate how the CBFE agent exhibits an epistemic drive, and actively plans ahead to account for the impact of predicted outcomes. Compared to an EFE agent, the CBFE agent incurs expected reward in significantly more environmental scenarios. We conclude that CBFE optimization by message passing suggests a general mechanism for epistemic-aware AIF in free-form generative models.

READ FULL TEXT

page 24

page 26

page 29

research
06/05/2023

Realising Synthetic Active Inference Agents, Part II: Variational Message Updates

The Free Energy Principle (FEP) describes (biological) agents as minimis...
research
02/17/2021

Chance-Constrained Active Inference

Active Inference (ActInf) is an emerging theory that explains perception...
research
06/13/2023

Realising Synthetic Active Inference Agents, Part I: Epistemic Objectives and Graphical Specification Language

The Free Energy Principle (FEP) is a theoretical framework for describin...
research
04/17/2020

Whence the Expected Free Energy?

The Expected Free Energy (EFE) is a central quantity in the theory of ac...
research
04/23/2021

Realising Active Inference in Variational Message Passing: the Outcome-blind Certainty Seeker

Active inference is a state-of-the-art framework in neuroscience that of...
research
08/01/2023

Active Inference in String Diagrams: A Categorical Account of Predictive Processing and Free Energy

We present a categorical formulation of the cognitive frameworks of Pred...
research
12/26/2021

AIDA: An Active Inference-based Design Agent for Audio Processing Algorithms

In this paper we present AIDA, which is an active inference-based agent ...

Please sign up or login with your details

Forgot password? Click here to reset