DeepAI AI Chat
Log In Sign Up

ASQ-IT: Interactive Explanations for Reinforcement-Learning Agents

by   Yotam Amitai, et al.
University of Haifa

As reinforcement learning methods increasingly amass accomplishments, the need for comprehending their solutions becomes more crucial. Most explainable reinforcement learning (XRL) methods generate a static explanation depicting their developers' intuition of what should be explained and how. In contrast, literature from the social sciences proposes that meaningful explanations are structured as a dialog between the explainer and the explainee, suggesting a more active role for the user and her communication with the agent. In this paper, we present ASQ-IT – an interactive tool that presents video clips of the agent acting in its environment based on queries given by the user that describe temporal properties of behaviors of interest. Our approach is based on formal methods: queries in ASQ-IT's user interface map to a fragment of Linear Temporal Logic over finite traces (LTLf), which we developed, and our algorithm for query processing is based on automata theory. User studies show that end-users can understand and formulate queries in ASQ-IT, and that using ASQ-IT assists users in identifying faulty agent behaviors.


Interactive Explanations: Diagnosis and Repair of Reinforcement Learning Based Agent Behaviors

Reinforcement learning techniques successfully generate convincing agent...

Toward Policy Explanations for Multi-Agent Reinforcement Learning

Advances in multi-agent reinforcement learning (MARL) enable sequential ...

Decisions that Explain Themselves: A User-Centric Deep Reinforcement Learning Explanation System

With deep reinforcement learning (RL) systems like autonomous driving be...

A Framework for Understanding and Visualizing Strategies of RL Agents

Recent years have seen significant advances in explainable AI as the nee...

MarioMix: Creating Aligned Playstyles for Bots with Interactive Reinforcement Learning

In this paper, we propose a generic framework that enables game develope...

Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System

Recently, there has been increasing interest in transparency and interpr...