DeepAI AI Chat
Log In Sign Up

Free Energy and the Generalized Optimality Equations for Sequential Decision Making

by   Pedro A. Ortega, et al.
Max Planck Society

The free energy functional has recently been proposed as a variational principle for bounded rational decision-making, since it instantiates a natural trade-off between utility gains and information processing costs that can be axiomatically derived. Here we apply the free energy principle to general decision trees that include both adversarial and stochastic environments. We derive generalized sequential optimality equations that not only include the Bellman optimality equations as a limit case, but also lead to well-known decision-rules such as Expectimax, Minimax and Expectiminimax. We show how these decision-rules can be derived from a single free energy principle that assigns a resource parameter to each node in the decision tree. These resource parameters express a concrete computational cost that can be measured as the amount of samples that are needed from the distribution that belongs to each node. The free energy principle therefore provides the normative basis for generalized optimality equations that account for both adversarial and stochastic environments.


page 1

page 2

page 3

page 4


Information, Utility & Bounded Rationality

Perfectly rational decision-makers maximize expected utility, but crucia...

Bounded Rational Decision-Making in Changing Environments

A perfectly rational decision-maker chooses the best action with the hig...

Hierarchical State Abstractions for Decision-Making Problems with Computational Constraints

In this semi-tutorial paper, we first review the information-theoretic a...

Deformed Statistics Formulation of the Information Bottleneck Method

The theoretical basis for a candidate variational principle for the info...

Information-Theoretic Bounded Rationality

Bounded rationality, that is, decision-making and planning under resourc...

An Adversarial Interpretation of Information-Theoretic Bounded Rationality

Recently, there has been a growing interest in modeling planning with in...

α-Discounting Multi-Criteria Decision Making (α-D MCDM)

In this book we introduce a new procedure called α-Discounting Method fo...