Model-based Utility Functions

11/16/2011
by   Bill Hibbard, et al.
0

Orseau and Ring, as well as Dewey, have recently described problems, including self-delusion, with the behavior of agents using various definitions of utility functions. An agent's utility function is defined in terms of the agent's history of interactions with its environment. This paper argues, via two examples, that the behavior problems can be avoided by formulating the utility function in two steps: 1) inferring a model of the environment from interactions, and 2) computing utility as a function of the environment model. Basing a utility function on a model that the agent must learn implies that the utility function must initially be expressed in terms of specifications to be matched to structures in the learned model. These specifications constitute prior assumptions about the environment so this approach will not work with arbitrary environments. But the approach should work for agents designed by humans to act in the physical world. The paper also addresses the issue of self-modifying agents and shows that if provided with the possibility to modify their utility functions agents will not choose to do so, under some usual assumptions.

READ FULL TEXT
research
07/31/2009

Convergence of Expected Utility for Universal AI

We consider a sequence of repeated interactions between an agent and an ...
research
11/12/2020

Performance of Bounded-Rational Agents With the Ability to Self-Modify

Self-modification of agents embedded in complex environments is hard to ...
research
04/06/2023

Semantic Information in a model of Resource Gathering Agents

We explore the application of a new theory of Semantic Information to th...
research
05/31/2018

Agents and Devices: A Relative Definition of Agency

According to Dennett, the same system may be described using a `physical...
research
11/05/2014

Ethical Artificial Intelligence

This book-length article combines several peer reviewed papers and new m...
research
03/23/2022

Possibility Before Utility: Learning And Using Hierarchical Affordances

Reinforcement learning algorithms struggle on tasks with complex hierarc...
research
07/27/2011

Time Consistent Discounting

A possibly immortal agent tries to maximise its summed discounted reward...

Please sign up or login with your details

Forgot password? Click here to reset