Policy Consolidation for Continual Reinforcement Learning

02/01/2019
by   Christos Kaplanis, et al.
14

We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is agnostic to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in continuously changing environments. In our policy consolidation model, the policy network interacts with a cascade of hidden networks that simultaneously remember the agent's policy at a range of timescales and regularise the current policy by its own history, thereby improving its ability to learn without forgetting. We find that the model improves continual learning relative to baselines on a number of continuous control tasks in single-task, alternating two-task, and multi-agent competitive self-play settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/16/2020

Continual Reinforcement Learning with Multi-Timescale Replay

In this paper, we propose a multi-timescale replay (MTR) buffer for impr...
research
12/31/2021

Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments

A key challenge for AI is to build embodied systems that operate in dyna...
research
09/15/2021

Life-Long Multi-Task Learning of Adaptive Path Tracking Policy for Autonomous Vehicle

This paper proposes a life-long adaptive path tracking policy learning m...
research
06/21/2019

Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction

Deep reinforcement learning has made significant progress in the field o...
research
05/29/2023

Continual Task Allocation in Meta-Policy Network via Sparse Prompting

How to train a generalizable meta-policy by continually learning a seque...
research
11/14/2022

Hierarchically Structured Task-Agnostic Continual Learning

One notable weakness of current machine learning algorithms is the poor ...
research
06/05/2021

Same State, Different Task: Continual Reinforcement Learning without Interference

Continual Learning (CL) considers the problem of training an agent seque...

Please sign up or login with your details

Forgot password? Click here to reset