DeepAI AI Chat
Log In Sign Up

Evolved Policy Gradients

by   Rein Houthooft, et al.

We propose a meta-learning approach for learning gradient-based reinforcement learning (RL) algorithms. The idea is to evolve a differentiable loss function, such that an agent, which optimizes its policy to minimize this loss, will achieve high rewards. The loss is parametrized via temporal convolutions over the agent's experience. Because this loss is highly flexible in its ability to take into account the agent's history, it enables fast task learning and eliminates the need for reward shaping at test time. Empirical results show that our evolved policy gradient algorithm achieves faster learning on several randomized environments compared to an off-the-shelf policy gradient method. Moreover, at test time, our learner optimizes only its learned loss function, and requires no explicit reward signal. In effect, the agent internalizes the reward structure, suggesting a direction toward agents that learn to solve new tasks simply from intrinsic motivation.


On Learning Intrinsic Rewards for Policy Gradient Methods

In many sequential decision making tasks, it is challenging to design re...

Learning Transferable Reward for Query Object Localization with Policy Adaptation

We propose a reinforcement learning based approach to query object local...

Policy Gradients Incorporating the Future

Reasoning about the future – understanding how decisions in the present ...

Directed Policy Gradient for Safe Reinforcement Learning with Human Advice

Many currently deployed Reinforcement Learning agents work in an environ...

VIABLE: Fast Adaptation via Backpropagating Learned Loss

In few-shot learning, typically, the loss function which is applied at t...

Deep 360 Pilot: Learning a Deep Agent for Piloting through 360° Sports Video

Watching a 360 sports video requires a viewer to continuously select a v...

Meta-Learning via Learned Loss

We present a meta-learning approach based on learning an adaptive, high-...

Code Repositories


Open-sourced code for Evolved Policy Gradients

view repo