Hindsight Credit Assignment

12/05/2019
by   Anna Harutyunyan, et al.
0

We consider the problem of efficient credit assignment in reinforcement learning. In order to efficiently and meaningfully utilize new data, we propose to explicitly assign credit to past decisions based on the likelihood of them having led to the observed outcome. This approach uses new information in hindsight, rather than employing foresight. Somewhat surprisingly, we show that value functions can be rewritten through this lens, yielding a new family of algorithms. We study the properties of these algorithms, and empirically show that they successfully address important credit assignment challenges, through a set of illustrative tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/20/2022

Selective Credit Assignment

Efficient credit assignment is essential for reinforcement learning algo...
research
06/08/2021

Towards Practical Credit Assignment for Deep Reinforcement Learning

Credit assignment is a fundamental problem in reinforcement learning, th...
research
11/19/2019

Variance Reduced Advantage Estimation with δ Hindsight Credit Assignment

Hindsight Credit Assignment (HCA) refers to a recently proposed family o...
research
07/18/2019

Credit Assignment as a Proxy for Transfer in Reinforcement Learning

The ability to transfer representations to novel environments and tasks ...
research
06/01/2022

Predecessor Features

Any reinforcement learning system must be able to identify which past ev...
research
02/09/2022

A new perspective on classification: optimally allocating limited resources to uncertain tasks

A central problem in business concerns the optimal allocation of limited...
research
07/16/2023

Credit Assignment: Challenges and Opportunities in Developing Human-like AI Agents

Temporal credit assignment is crucial for learning and skill development...

Please sign up or login with your details

Forgot password? Click here to reset