Causal Reasoning from Meta-reinforcement Learning

01/23/2019
by   Ishita Dasgupta, et al.
0

Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents. Here we explore whether causal reasoning can emerge via meta-reinforcement learning. We train a recurrent network with model-free reinforcement learning to solve a range of problems that each contain causal structure. We find that the trained agent can perform causal reasoning in novel situations in order to obtain rewards. The agent can select informative interventions, draw causal inferences from observational data, and make counterfactual predictions. Although established formal causal reasoning algorithms also exist, in this paper we show that such reasoning can arise from model-free reinforcement learning, and suggest that causal reasoning in complex settings may benefit from the more end-to-end learning-based approaches presented here. This work also offers new strategies for structured exploration in reinforcement learning, by providing agents with the ability to perform -- and interpret -- experiments.

READ FULL TEXT

page 5

page 6

page 7

research
05/26/2020

Towards intervention-centric causal reasoning in learning agents

Interventions are central to causal learning and reasoning. Yet ultimate...
research
08/20/2019

Reinforcement Learning is not a Causal problem

We use an analogy between non-isomorphic mathematical structures defined...
research
07/18/2022

A Meta-Reinforcement Learning Algorithm for Causal Discovery

Causal discovery is a major task with the utmost importance for machine ...
research
05/17/2023

Pragmatic Reasoning in Structured Signaling Games

In this work we introduce a structured signaling game, an extension of t...
research
10/03/2019

Causal Induction from Visual Observations for Goal Directed Tasks

Causal reasoning has been an indispensable capability for humans and oth...
research
07/06/2020

Counterfactual Data Augmentation using Locally Factored Dynamics

Many dynamic processes, including common scenarios in robotic control an...
research
12/07/2021

Tell me why! – Explanations support learning of relational and causal structure

Explanations play a considerable role in human learning, especially in a...

Please sign up or login with your details

Forgot password? Click here to reset