DeepAI AI Chat
Log In Sign Up

Decisions that Explain Themselves: A User-Centric Deep Reinforcement Learning Explanation System

by   Xiaoran Wu, et al.

With deep reinforcement learning (RL) systems like autonomous driving being wildly deployed but remaining largely opaque, developers frequently use explainable RL (XRL) tools to better understand and work with deep RL agents. However, previous XRL works employ a techno-centric research approach, ignoring how RL developers perceive the generated explanations. Through a pilot study, we identify major goals for RL practitioners to use XRL methods and four pitfalls that widen the gap between existing XRL methods and these goals. The pitfalls include inaccessible reasoning processes, inconsistent or unintelligible explanations, and explanations that cannot be generalized. To fill the discovered gap, we propose a counterfactual-inference-based explanation method that discovers the details of the reasoning process of RL agents and generates natural language explanations. Surrounding this method, we build an interactive XRL system where users can actively explore explanations and influential information. In a user study with 14 participants, we validated that developers identified 20.9 agents with our system compared to the baseline method, and using our system helped end users improve their performance in actionability tests by 25.1 an auto-driving task and by 16.9


page 1

page 7

page 12


Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning

Counterfactual explanations, which deal with "why not?" scenarios, can p...

Generating Explanations from Deep Reinforcement Learning Using Episodic Memory

Deep Reinforcement Learning (RL) involves the use of Deep Neural Network...

Identifying Reasoning Flaws in Planning-Based RL Using Tree Explanations

Enabling humans to identify potential flaws in an agent's decision makin...

Counterfactual Explanations for Reinforcement Learning

While AI algorithms have shown remarkable success in various fields, the...

ASQ-IT: Interactive Explanations for Reinforcement-Learning Agents

As reinforcement learning methods increasingly amass accomplishments, th...

Distal Explanations for Explainable Reinforcement Learning Agents

Causal explanations present an intuitive way to understand the course of...

Explainable Multi-Agent Reinforcement Learning for Temporal Queries

As multi-agent reinforcement learning (MARL) systems are increasingly de...