Methodical Advice Collection and Reuse in Deep Reinforcement Learning

04/14/2022
by   Sahir, et al.
19

Reinforcement learning (RL) has shown great success in solving many challenging tasks via use of deep neural networks. Although using deep learning for RL brings immense representational power, it also causes a well-known sample-inefficiency problem. This means that the algorithms are data-hungry and require millions of training samples to converge to an adequate policy. One way to combat this issue is to use action advising in a teacher-student framework, where a knowledgeable teacher provides action advice to help the student. This work considers how to better leverage uncertainties about when a student should ask for advice and if the student can model the teacher to ask for less advice. The student could decide to ask for advice when it is uncertain or when both it and its model of the teacher are uncertain. In addition to this investigation, this paper introduces a new method to compute uncertainty for a deep RL agent using a secondary neural network. Our empirical results show that using dual uncertainties to drive advice collection and reuse may improve learning performance across several Atari games.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/31/2022

Teacher-student curriculum learning for reinforcement learning

Reinforcement learning (rl) is a popular paradigm for sequential decisio...
research
09/15/2021

DCUR: Data Curriculum for Teaching via Samples with Reinforcement Learning

Deep reinforcement learning (RL) has shown great empirical successes, bu...
research
06/12/2020

Mutual Information Based Knowledge Transfer Under State-Action Dimension Mismatch

Deep reinforcement learning (RL) algorithms have achieved great success ...
research
04/17/2021

Learning on a Budget via Teacher Imitation

Deep Reinforcement Learning (RL) techniques can benefit greatly from lev...
research
04/17/2021

Action Advising with Advice Imitation in Deep Reinforcement Learning

Action advising is a peer-to-peer knowledge exchange technique built on ...
research
07/06/2023

TGRL: An Algorithm for Teacher Guided Reinforcement Learning

Learning from rewards (i.e., reinforcement learning or RL) and learning ...
research
10/01/2020

Student-Initiated Action Advising via Advice Novelty

Action advising is a knowledge exchange mechanism between peers, namely ...

Please sign up or login with your details

Forgot password? Click here to reset