Shaped Policy Search for Evolutionary Strategies using Waypoints

05/30/2021
by   Kiran Lekkala, et al.
0

In this paper, we try to improve exploration in Blackbox methods, particularly Evolution strategies (ES), when applied to Reinforcement Learning (RL) problems where intermediate waypoints/subgoals are available. Since Evolutionary strategies are highly parallelizable, instead of extracting just a scalar cumulative reward, we use the state-action pairs from the trajectories obtained during rollouts/evaluations, to learn the dynamics of the agent. The learnt dynamics are then used in the optimization procedure to speed-up training. Lastly, we show how our proposed approach is universally applicable by presenting results from experiments conducted on Carla driving and UR5 robotic arm simulators.

READ FULL TEXT
research
06/06/2017

Parameter Space Noise for Exploration

Deep reinforcement learning (RL) methods generally engage in exploratory...
research
02/08/2019

Novelty Search for Deep Reinforcement Learning Policy Network Weights by Action Sequence Edit Metric Distance

Reinforcement learning (RL) problems often feature deceptive local optim...
research
10/01/2021

Guiding Evolutionary Strategies by Differentiable Robot Simulators

In recent years, Evolutionary Strategies were actively explored in robot...
research
05/17/2018

Evolutionary RL for Container Loading

Loading the containers on the ship from a yard, is an impor- tant part o...
research
05/10/2023

Supplementing Gradient-Based Reinforcement Learning with Simple Evolutionary Ideas

We present a simple, sample-efficient algorithm for introducing large bu...
research
07/19/2017

Reinforcement Learning Produces Dominant Strategies for the Iterated Prisoner's Dilemma

We present tournament results and several powerful strategies for the It...
research
05/05/2023

Biophysical Cybernetics of Directed Evolution and Eco-evolutionary Dynamics

Many major questions in the theory of evolutionary dynamics can in a mea...

Please sign up or login with your details

Forgot password? Click here to reset