Fast Online Adaptation in Robotics through Meta-Learning Embeddings of Simulated Priors

03/10/2020
by   Rituraj Kaushik, et al.
12

Meta-learning algorithms can accelerate the model-based reinforcement learning (MBRL) algorithms by finding an initial set of parameters for the dynamical model such that the model can be trained to match the actual dynamics of the system with only a few data-points. However, in the real world, a robot might encounter any situation starting from motor failures to finding itself in a rocky terrain where the dynamics of the robot can be significantly different from one another. In this paper, first, we show that when meta-training situations (the prior situations) have such diverse dynamics, using a single set of meta-trained parameters as a starting point still requires a large number of observations from the real system to learn a useful model of the dynamics. Second, we propose an algorithm called FAMLE that mitigates this limitation by meta-training several initial starting points (i.e., initial parameters) for training the model and allows the robot to select the most suitable starting point to adapt the model to the current situation with only a few gradient steps. We compare FAMLE to MBRL, MBRL with a meta-trained model with MAML, and model-free policy search algorithm PPO for various simulated and real robotic tasks, and show that FAMLE allows the robots to adapt to novel damages in significantly fewer time-steps than the baselines.

READ FULL TEXT
research
03/02/2020

Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning

Learning adaptable policies is crucial for robots to operate autonomousl...
research
07/16/2019

Adaptive Prior Selection for Repertoire-based Online Learning in Robotics

Among the data-efficient approaches for online adaptation in robotics (m...
research
03/05/2021

Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms

Reinforcement learning methods can achieve significant performance but r...
research
05/11/2021

A Meta Reinforcement Learning-based Approach for Self-Adaptive System

A self-learning adaptive system (SLAS) uses machine learning to enable a...
research
09/28/2019

Learning Fast Adaptation with Meta Strategy Optimization

The ability to walk in new scenarios is a key milestone on the path towa...
research
09/21/2020

Adaptive Meta-Learning for Identification of Rover-Terrain Dynamics

Rovers require knowledge of terrain to plan trajectories that maximize s...
research
08/31/2018

The NEU Meta-Algorithm for Geometric Learning with Applications in Finance

We introduce a meta-algorithm, called non-Euclidean upgrading (NEU), whi...

Please sign up or login with your details

Forgot password? Click here to reset