Forward-Backward Reinforcement Learning

03/27/2018
by   Ashley D. Edwards, et al.
0

Goals for reinforcement learning problems are typically defined through hand-specified rewards. To design such problems, developers of learning algorithms must inherently be aware of what the task goals are, yet we often require agents to discover them on their own without any supervision beyond these sparse rewards. While much of the power of reinforcement learning derives from the concept that agents can learn with little guidance, this requirement greatly burdens the training process. If we relax this one restriction and endow the agent with knowledge of the reward function, and in particular of the goal, we can leverage backwards induction to accelerate training. To achieve this, we propose training a model to learn to take imagined reversal steps from known goal states. Rather than training an agent exclusively to determine how to reach a goal while moving forwards in time, our approach travels backwards to jointly predict how we got there. We evaluate our work in Gridworld and Towers of Hanoi and empirically demonstrate that it yields better performance than standard DDQN.

READ FULL TEXT
research
05/17/2017

Automatic Goal Generation for Reinforcement Learning Agents

Reinforcement learning is a powerful technique to train an agent to perf...
research
11/21/2019

Accelerating Reinforcement Learning with Suboptimal Guidance

Reinforcement Learning in domains with sparse rewards is a difficult pro...
research
12/22/2016

First-Person Activity Forecasting with Online Inverse Reinforcement Learning

We address the problem of incrementally modeling and forecasting long-te...
research
07/17/2018

Reinforcement Learning for LTLf/LDLf Goals

MDPs extended with LTLf/LDLf non-Markovian rewards have recently attract...
research
02/20/2023

Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoning

We approach the problem of understanding how people interact with each o...
research
10/21/2014

Where do goals come from? A Generic Approach to Autonomous Goal-System Development

Goals express agents' intentions and allow them to organize their behavi...
research
05/05/2021

Solving Sokoban with backward reinforcement learning

In some puzzles, the strategy we need to use near the goal can be quite ...

Please sign up or login with your details

Forgot password? Click here to reset