Metaoptimization on a Distributed System for Deep Reinforcement Learning

02/07/2019
by   Greg Heinrich, et al.
0

Training intelligent agents through reinforcement learning is a notoriously unstable procedure. Massive parallelization on GPUs and distributed systems has been exploited to generate a large amount of training experiences and consequently reduce instabilities, but the success of training remains strongly influenced by the choice of the hyperparameters. To overcome this issue, we introduce HyperTrick, a new metaoptimization algorithm, and show its effective application to tune hyperparameters in the case of deep reinforcement learning, while learning to play different Atari games on a distributed system. Our analysis provides evidence of the interaction between the identification of the optimal hyperparameters and the learned policy, that is typical of the case of metaoptimization for deep reinforcement learning. When compared with state-of-the-art metaoptimization algorithms, HyperTrick is characterized by a simpler implementation and it allows learning similar policies, while making a more effective use of the computational resources in a distributed system.

READ FULL TEXT

page 4

page 9

research
07/15/2015

Massively Parallel Methods for Deep Reinforcement Learning

We present the first massively distributed architecture for deep reinfor...
research
08/23/2020

DSP: A Differential Spatial Prediction Scheme for Comprehensive real industrial datasets

Inverse Distance Weighted models (IDW) have been widely used for predict...
research
01/09/2018

Distributed Deep Reinforcement Learning: Learn how to play Atari games in 21 minutes

We present a study in Distributed Deep Reinforcement Learning (DDRL) foc...
research
03/20/2022

MicroRacer: a didactic environment for Deep Reinforcement Learning

MicroRacer is a simple, open source environment inspired by car racing e...
research
09/20/2019

Bayesian Optimization for Iterative Learning

The success of deep (reinforcement) learning systems crucially depends o...
research
05/02/2019

Collaborative Evolutionary Reinforcement Learning

Deep reinforcement learning algorithms have been successfully applied to...
research
11/20/2020

Revisiting Rainbow: Promoting more insightful and inclusive deep reinforcement learning research

Since the introduction of DQN, a vast majority of reinforcement learning...

Please sign up or login with your details

Forgot password? Click here to reset