DeepAI AI Chat
Log In Sign Up

Tell me why! – Explanations support learning of relational and causal structure

by   Andrew K. Lampinen, et al.

Explanations play a considerable role in human learning, especially in areas that remain major challenges for AI – forming abstractions, and learning about the relational and causal structure of the world. Here, we explore whether reinforcement learning agents might likewise benefit from explanations. We outline a family of relational tasks that involve selecting an object that is the odd one out in a set (i.e., unique along one of many possible feature dimensions). Odd-one-out tasks require agents to reason over multi-dimensional relationships among a set of objects. We show that agents do not learn these tasks well from reward alone, but achieve >90 trained to generate language explaining object properties or why a choice is correct or incorrect. In further experiments, we show how predicting explanations enables agents to generalize appropriately from ambiguous, causally-confounded training, and even to meta-learn to perform experimental interventions to identify causal structure. We show that explanations help overcome the tendency of agents to fixate on simple features, and explore which aspects of explanations make them most beneficial. Our results suggest that learning from explanations is a powerful principle that could offer a promising path towards training more robust and general machine learning systems.


Explainable Reinforcement Learning Through a Causal Lens

Prevalent theories in cognitive science propose that humans understand a...

Causal Analysis of Agent Behavior for AI Safety

As machine learning systems become more powerful they also become increa...

Explainability Via Causal Self-Talk

Explaining the behavior of AI systems is an important problem that, in p...

Passive learning of active causal strategies in agents and language models

What can be learned about causality and experimentation from passive dat...

A Closer Look at Reward Decomposition for High-Level Robotic Explanations

Explaining the behavior of intelligent agents such as robots to humans i...

Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability

Explaining AI systems is fundamental both to the development of high per...