A Model-based Approach for Sample-efficient Multi-task Reinforcement Learning

07/11/2019
by   Nicholas C. Landolfi, et al.
7

The aim of multi-task reinforcement learning is two-fold: (1) efficiently learn by training against multiple tasks and (2) quickly adapt, using limited samples, to a variety of new tasks. In this work, the tasks correspond to reward functions for environments with the same (or similar) dynamical models. We propose to learn a dynamical model during the training process and use this model to perform sample-efficient adaptation to new tasks at test time. Our algorithm sequentially trains against several tasks. We use significantly fewer samples by performing policy optimization only in a "virtual" environment whose transitions are given by our learned dynamical model. Upon encountering a new task, we first warm-up a policy on our learned dynamical model, which requires no new samples from the environment. We then adapt the dynamical model with samples from this policy in the real environment. We evaluate our approach on several continuous control benchmarks and demonstrate its efficacy over MAML, a state-of-the-art meta-learning algorithm, on these tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/16/2020

Model-based Adversarial Meta-Reinforcement Learning

Meta-reinforcement learning (meta-RL) aims to learn from multiple traini...
research
03/30/2018

Learning to Adapt: Meta-Learning for Model-Based Control

Although reinforcement learning methods can achieve impressive results i...
research
05/11/2021

A Meta Reinforcement Learning-based Approach for Self-Adaptive System

A self-learning adaptive system (SLAS) uses machine learning to enable a...
research
03/07/2016

Learning Shared Representations in Multi-task Reinforcement Learning

We investigate a paradigm in multi-task reinforcement learning (MT-RL) i...
research
06/14/2020

Provably Efficient Model-based Policy Adaptation

The high sample complexity of reinforcement learning challenges its use ...
research
04/05/2022

Model Based Meta Learning of Critics for Policy Gradients

Being able to seamlessly generalize across different tasks is fundamenta...
research
10/25/2021

Multitask Adaptation by Retrospective Exploration with Learned World Models

Model-based reinforcement learning (MBRL) allows solving complex tasks i...

Please sign up or login with your details

Forgot password? Click here to reset