Pragmatic-Pedagogic Value Alignment

by   Jaime F. Fisac, et al.
berkeley college

For an autonomous system to provide value (e.g., to customers, designers, or society at large) it must have a reliable method to determine the intended goal. This is the essence of the value-alignment problem: ensuring that the objectives of an autonomous system match those of its human users. In robotics, value alignment is crucial to the design of collaborative robots that can integrate into human workflows, successfully learning and adapting to the objectives of their users as they go. We argue that a meaningful solution to the value-alignment problem will combine multi-agent decision theory with rich mathematical models of human cognition, enabling robots to tap into people's natural collaborative capabilities. We present a solution to the cooperative inverse reinforcement learning (CIRL) dynamic game using well-established models of decision making and theory of mind from cognitive science. The solution accounts for two crucial aspects of collaborative value alignment: that the human will not plan her actions in isolation, but will reason pedagogically about how the robot might learn from them; and that the robot should anticipate this and interpret the human's actions pragmatically. To our knowledge, this constitutes the first equilibrium analysis of value alignment grounded in an empirically validated cognitive model of the human.


Cooperative Inverse Reinforcement Learning

For an autonomous system to be helpful to humans and to pose no unwarran...

Cognitive science as a source of forward and inverse models of human decisions for robotics and control

Those designing autonomous systems that interact with humans will invari...

Goal Alignment: A Human-Aware Account of Value Alignment Problem

Value alignment problems arise in scenarios where the specified objectiv...

An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning

Our goal is for AI systems to correctly identify and act according to th...

Mammalian Value Systems

Characterizing human values is a topic deeply interwoven with the scienc...

Hierarchical Planning and Policy Shaping Shared Autonomy for Articulated Robots

In this work, we propose a novel shared autonomy framework to operate ar...

Please sign up or login with your details

Forgot password? Click here to reset