Hindsight Credit Assignment

by   Anna Harutyunyan, et al.

We consider the problem of efficient credit assignment in reinforcement learning. In order to efficiently and meaningfully utilize new data, we propose to explicitly assign credit to past decisions based on the likelihood of them having led to the observed outcome. This approach uses new information in hindsight, rather than employing foresight. Somewhat surprisingly, we show that value functions can be rewritten through this lens, yielding a new family of algorithms. We study the properties of these algorithms, and empirically show that they successfully address important credit assignment challenges, through a set of illustrative tasks.


page 1

page 2

page 3

page 4


Selective Credit Assignment

Efficient credit assignment is essential for reinforcement learning algo...

Towards Practical Credit Assignment for Deep Reinforcement Learning

Credit assignment is a fundamental problem in reinforcement learning, th...

Variance Reduced Advantage Estimation with δ Hindsight Credit Assignment

Hindsight Credit Assignment (HCA) refers to a recently proposed family o...

Credit Assignment as a Proxy for Transfer in Reinforcement Learning

The ability to transfer representations to novel environments and tasks ...

Predecessor Features

Any reinforcement learning system must be able to identify which past ev...

A new perspective on classification: optimally allocating limited resources to uncertain tasks

A central problem in business concerns the optimal allocation of limited...

Credit Assignment: Challenges and Opportunities in Developing Human-like AI Agents

Temporal credit assignment is crucial for learning and skill development...

Please sign up or login with your details

Forgot password? Click here to reset