Transparency and Explanation in Deep Reinforcement Learning Neural Networks

by   Rahul Iyer, et al.

Autonomous AI systems will be entering human society in the near future to provide services and work alongside humans. For those systems to be accepted and trusted, the users should be able to understand the reasoning process of the system, i.e. the system should be transparent. System transparency enables humans to form coherent explanations of the system's decisions and actions. Transparency is important not only for user trust, but also for software debugging and certification. In recent years, Deep Neural Networks have made great advances in multiple application areas. However, deep neural networks are opaque. In this paper, we report on work in transparency in Deep Reinforcement Learning Networks (DRLN). Such networks have been extremely successful in accurately learning action control in image input domains, such as Atari games. In this paper, we propose a novel and general method that (a) incorporates explicit object recognition processing into deep reinforcement learning models, (b) forms the basis for the development of "object saliency maps", to provide visualization of internal states of DRLNs, thus enabling the formation of explanations and (c) can be incorporated in any existing deep reinforcement learning framework. We present computational results and human experiments to evaluate our approach.


page 4

page 5


Object-sensitive Deep Reinforcement Learning

Deep reinforcement learning has become popular over recent years, showin...

Explainable Deep Reinforcement Learning for UAV Autonomous Navigation

Modern deep reinforcement learning plays an important role to solve a wi...

Deep Reinforcement Learning Models Predict Visual Responses in the Brain: A Preliminary Result

Supervised deep convolutional neural networks (DCNNs) are currently one ...

DCNFIS: Deep Convolutional Neuro-Fuzzy Inference System

A key challenge in eXplainable Artificial Intelligence is the well-known...

DAX: Deep Argumentative eXplanation for Neural Networks

Despite the rapid growth in attention on eXplainable AI (XAI) of late, e...

Approximating a deep reinforcement learning docking agent using linear model trees

Deep reinforcement learning has led to numerous notable results in robot...

Unbiased Deep Reinforcement Learning: A General Training Framework for Existing and Future Algorithms

In recent years deep neural networks have been successfully applied to t...

Please sign up or login with your details

Forgot password? Click here to reset