Which Experiences Are Influential for Your Agent? Policy Iteration with Turn-over Dropout

01/26/2023
by   Takuya Hiraoka, et al.
0

In reinforcement learning (RL) with experience replay, experiences stored in a replay buffer influence the RL agent's performance. Information about the influence is valuable for various purposes, including experience cleansing and analysis. One method for estimating the influence of individual experiences is agent comparison, but it is prohibitively expensive when there is a large number of experiences. In this paper, we present PI+ToD as a method for efficiently estimating the influence of experiences. PI+ToD is a policy iteration that efficiently estimates the influence of experiences by utilizing turn-over dropout. We demonstrate the efficiency of PI+ToD with experiments in MuJoCo environments.

READ FULL TEXT

page 7

page 8

page 14

page 15

page 16

page 17

research
09/29/2020

Lucid Dreaming for Experience Replay: Refreshing Past States with the Current Policy

Experience replay (ER) improves the data efficiency of off-policy reinfo...
research
03/15/2022

Zipfian environments for Reinforcement Learning

As humans and animals learn in the natural world, they encounter distrib...
research
08/24/2020

Improved Memories Learning

We propose Improved Memories Learning (IMeL), a novel algorithm that tur...
research
02/05/2021

Revisiting Prioritized Experience Replay: A Value Perspective

Experience replay enables off-policy reinforcement learning (RL) agents ...
research
09/06/2018

ARCHER: Aggressive Rewards to Counter bias in Hindsight Experience Replay

Experience replay is an important technique for addressing sample-ineffi...
research
02/17/2022

Retrieval-Augmented Reinforcement Learning

Most deep reinforcement learning (RL) algorithms distill experience into...
research
01/03/2017

Experiences with Some Benchmarks for Deductive Databases and Implementations of Bottom-Up Evaluation

OpenRuleBench is a large benchmark suite for rule engines, which include...

Please sign up or login with your details

Forgot password? Click here to reset