Sample Complexity of Multi-task Reinforcement Learning

09/26/2013
by   Emma Brunskill, et al.
0

Transferring knowledge across a sequence of reinforcement-learning tasks is challenging, and has a number of important applications. Though there is encouraging empirical evidence that transfer can improve performance in subsequent reinforcement-learning tasks, there has been very little theoretical analysis. In this paper, we introduce a new multi-task algorithm for a sequence of reinforcement-learning tasks when each task is sampled independently from (an unknown) distribution over a finite set of Markov decision processes whose parameters are initially unknown. For this setting, we prove under certain assumptions that the per-task sample complexity of exploration is reduced significantly due to transfer compared to standard single-task algorithms. Our multi-task algorithm also has the desired characteristic that it is guaranteed not to exhibit negative transfer: in the worst case its per-task sample complexity is comparable to the corresponding single-task algorithm.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/10/2015

The Online Coupon-Collector Problem and Its Application to Lifelong Reinforcement Learning

Transferring knowledge across a sequence of related tasks is an importan...
research
07/14/2020

Multi-Task Reinforcement Learning as a Hidden-Parameter Block MDP

Multi-task reinforcement learning is a rich paradigm where information f...
research
07/05/2019

Attentive Multi-Task Deep Reinforcement Learning

Sharing knowledge between tasks is vital for efficient learning in a mul...
research
10/27/2021

Provable Lifelong Learning of Representations

In lifelong learning, the tasks (or classes) to be learned arrive sequen...
research
02/16/2020

TempLe: Learning Template of Transitions for Sample Efficient Multi-task RL

Transferring knowledge among various environments is important to effici...
research
06/02/2023

Efficient Multi-Task and Transfer Reinforcement Learning with Parameter-Compositional Framework

In this work, we investigate the potential of improving multi-task train...
research
02/26/2020

Generalized Hindsight for Reinforcement Learning

One of the key reasons for the high sample complexity in reinforcement l...

Please sign up or login with your details

Forgot password? Click here to reset