Strategy Discovery and Mixture in Lifelong Learning from Heterogeneous Demonstration

02/14/2022
by   Sravan Jayanthi, et al.
0

Learning from Demonstration (LfD) approaches empower end-users to teach robots novel tasks via demonstrations of the desired behaviors, democratizing access to robotics. A key challenge in LfD research is that users tend to provide heterogeneous demonstrations for the same task due to various strategies and preferences. Therefore, it is essential to develop LfD algorithms that ensure flexibility (the robot adapts to personalized strategies), efficiency (the robot achieves sample-efficient adaptation), and scalability (robot reuses a concise set of strategies to represent a large amount of behaviors). In this paper, we propose a novel algorithm, Dynamic Multi-Strategy Reward Distillation (DMSRD), which distills common knowledge between heterogeneous demonstrations, leverages learned strategies to construct mixture policies, and continues to improve by learning from all available data. Our personalized, federated, and lifelong LfD architecture surpasses benchmarks in two continuous control problems with an average 77% improvement in policy returns and 42% improvement in log likelihood, alongside stronger task reward correlation and more precise strategy rewards.

READ FULL TEXT
research
09/24/2022

Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations

Learning from Demonstration (LfD) approaches empower end-users to teach ...
research
03/25/2022

Quantifying Demonstration Quality for Robot Learning and Generalization

Learning from Demonstration (LfD) seeks to democratize robotics by enabl...
research
07/12/2023

Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation

Policies often fail due to distribution shift – changes in the state and...
research
04/12/2022

Learning Performance Graphs from Demonstrations via Task-Based Evaluations

In the learning from demonstration (LfD) paradigm, understanding and eva...
research
03/31/2021

Ergodic imitation: Learning from what to do and what not to do

With growing access to versatile robotics, it is beneficial for end user...
research
07/10/2021

Informing Real-time Corrections in Corrective Shared Autonomy Through Expert Demonstrations

Corrective Shared Autonomy is a method where human corrections are layer...
research
06/12/2022

Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policies

In this paper, we present a novel Heavy-Tailed Stochastic Policy Gradien...

Please sign up or login with your details

Forgot password? Click here to reset