Explaining Reward Functions to Humans for Better Human-Robot Collaboration

10/08/2021
by   Lindsay Sanneman, et al.
0

Explainable AI techniques that describe agent reward functions can enhance human-robot collaboration in a variety of settings. One context where human understanding of agent reward functions is particularly beneficial is in the value alignment setting. In the value alignment context, an agent aims to infer a human's reward function through interaction so that it can assist the human with their tasks. If the human can understand where gaps exist in the agent's reward understanding, they will be able to teach more efficiently and effectively, leading to quicker human-agent team performance improvements. In order to support human collaborators in the value alignment setting and similar contexts, it is first important to understand the effectiveness of different reward explanation techniques in a variety of domains. In this paper, we introduce a categorization of information modalities for reward explanation techniques, suggest a suite of assessment techniques for human reward understanding, and introduce four axes of domain complexity. We then propose an experiment to study the relative efficacy of a broad set of reward explanation techniques covering multiple modalities of information in a set of domains of varying complexity.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/22/2021

Trust as Extended Control: Active Inference and User Feedback During Human-Robot Collaboration

To interact seamlessly with robots, users must infer the causes of a rob...
research
09/11/2023

Effect of Adapting to Human Preferences on Trust in Human-Robot Teaming

We present the effect of adapting to human preferences on trust in a hum...
research
06/11/2018

An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning

Our goal is for AI systems to correctly identify and act according to th...
research
12/02/2020

Value Alignment Verification

As humans interact with autonomous agents to perform increasingly compli...
research
11/12/2021

Human irrationality: both bad and good for reward inference

Assuming humans are (approximately) rational enables robots to infer rew...
research
06/16/2022

How to talk so your robot will learn: Instructions, descriptions, and pragmatics

From the earliest years of our lives, humans use language to express our...
research
06/30/2020

Mitigating undesirable emergent behavior arising between driver and semi-automated vehicle

Emergent behavior arising in a joint human-robot system cannot be fully ...

Please sign up or login with your details

Forgot password? Click here to reset