Including Uncertainty when Learning from Human Corrections

06/06/2018
by   Dylan P. Losey, et al.
0

It is difficult for humans to efficiently teach robots how to correctly perform a task. One intuitive solution is for the robot to iteratively learn the human's preferences from corrections, where the human improves the robot's current behavior at each iteration. When learning from corrections, we argue that while the robot should estimate the most likely human preferences, it should also know what it does not know, and integrate this uncertainty when making decisions. We advance the state-of-the-art by introducing a Kalman filter for learning from corrections: this approach also maintains the uncertainty of the estimated human preferences. Next, we demonstrate how uncertainty can be leveraged for active learning and risk-sensitive deployment. Our results indicate that maintaining and leveraging uncertainty leads to faster learning from human corrections.

READ FULL TEXT

page 5

page 6

research
02/15/2020

Let Me At Least Learn What You Really Like: Dealing With Noisy Humans When Learning Preferences

Learning the preferences of a human improves the quality of the interact...
research
05/28/2017

Should Robots be Obedient?

Intuitively, obedience -- following the order that a human gives -- seem...
research
05/19/2023

Risk-Sensitive Extended Kalman Filter

In robotics, designing robust algorithms in the face of estimation uncer...
research
06/19/2021

Learning the Preferences of Uncertain Humans with Inverse Decision Theory

Existing observational approaches for learning human preferences, such a...
research
03/09/2019

Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learning

It is incredibly easy for a system designer to misspecify the objective ...
research
05/26/2017

Risk-Sensitive Cooperative Games for Human-Machine Systems

Autonomous systems can substantially enhance a human's efficiency and ef...
research
03/27/2021

Two-Stage Clustering of Human Preferences for Action Prediction in Assembly Tasks

To effectively assist human workers in assembly tasks a robot must proac...

Please sign up or login with your details

Forgot password? Click here to reset