Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System

by   Xinzhi Wang, et al.

Recently, there has been increasing interest in transparency and interpretability in Deep Reinforcement Learning (DRL) systems. Verbal explanations, as the most natural way of communication in our daily life, deserve more attention, since they allow users to gain a better understanding of the system which ultimately could lead to a high level of trust and smooth collaboration. This paper reports a novel work in generating verbal explanations for DRL behaviors agent. A rule-based model is designed to construct explanations using a series of rules which are predefined with prior knowledge. A learning model is then proposed to expand the implicit logic of generating verbal explanation to general situations by employing rule-based explanations as training data. The learning model is shown to have better flexibility and generalizability than the static rule-based model. The performance of both models is evaluated quantitatively through objective metrics. The results show that verbal explanation generated by both models improve subjective satisfaction of users towards the interpretability of DRL systems. Additionally, seven variants of the learning model are designed to illustrate the contribution of input channels, attention mechanism, and proposed encoder in improving the quality of verbal explanation.


Toward Policy Explanations for Multi-Agent Reinforcement Learning

Advances in multi-agent reinforcement learning (MARL) enable sequential ...

Measurably Stronger Explanation Reliability via Model Canonization

While rule-based attribution methods have proven useful for providing lo...

An Explanation Mechanism for Bayesian Inferencing Systems

Explanation facilities are a particularly important feature of expert sy...

What Did You Think Would Happen? Explaining Agent Behaviour Through Intended Outcomes

We present a novel form of explanation for Reinforcement Learning, based...

Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Driving

Automated driving in urban settings is challenging chiefly due to the in...

Multi-focus Attention Network for Efficient Deep Reinforcement Learning

Deep reinforcement learning (DRL) has shown incredible performance in le...