Deep In-GPU Experience Replay

01/09/2018
by   Ben Parr, et al.
0

Experience replay allows a reinforcement learning agent to train on samples from a large amount of the most recent experiences. A simple in-RAM experience replay stores these most recent experiences in a list in RAM, and then copies sampled batches to the GPU for training. I moved this list to the GPU, thus creating an in-GPU experience replay, and a training step that no longer has inputs copied from the CPU. I trained an agent to play Super Smash Bros. Melee, using internal game memory values as inputs and outputting controller button presses. A single state in Melee contains 27 floats, so the full experience replay fits on a single GPU. For a batch size of 128, the in-GPU experience replay trained twice as fast as the in-RAM experience replay. As far as I know, this is the first in-GPU implementation of experience replay. Finally, I note a few ideas for fitting the experience replay inside the GPU when the environment state requires more memory.

READ FULL TEXT
research
07/08/2020

Double Prioritized State Recycled Experience Replay

Experience replay enables online reinforcement learning agents to store ...
research
02/04/2020

Bootstrapping a DQN Replay Memory with Synthetic Experiences

An important component of many Deep Reinforcement Learning algorithms is...
research
05/15/2018

Advances in Experience Replay

This project combines recent advances in experience replay techniques, n...
research
10/18/2017

The Effects of Memory Replay in Reinforcement Learning

Experience replay is a key technique behind many recent advances in deep...
research
07/16/2022

Associative Memory Based Experience Replay for Deep Reinforcement Learning

Experience replay is an essential component in deep reinforcement learni...
research
06/24/2019

Optimal Use of Experience in First Person Shooter Environments

Although reinforcement learning has made great strides recently, a conti...
research
10/04/2022

TripleE: Easy Domain Generalization via Episodic Replay

Learning how to generalize the model to unseen domains is an important a...

Please sign up or login with your details

Forgot password? Click here to reset