Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning

by   Hidenori Itaya, et al.

Deep reinforcement learning (DRL) has great potential for acquiring the optimal action in complex environments such as games and robot control. However, it is difficult to analyze the decision-making of the agent, i.e., the reasons it selects the action acquired by learning. In this work, we propose Mask-Attention A3C (Mask A3C), which introduces an attention mechanism into Asynchronous Advantage Actor-Critic (A3C), which is an actor-critic-based DRL method, and can analyze the decision-making of an agent in DRL. A3C consists of a feature extractor that extracts features from an image, a policy branch that outputs the policy, and a value branch that outputs the state value. In this method, we focus on the policy and value branches and introduce an attention mechanism into them. The attention mechanism applies a mask processing to the feature maps of each branch using mask-attention that expresses the judgment reason for the policy and state value with a heat map. We visualized mask-attention maps for games on the Atari 2600 and found we could easily analyze the reasons behind an agent's decision-making in various game tasks. Furthermore, experimental results showed that the agent could achieve a higher performance by introducing the attention mechanism.


page 5

page 9

page 11

page 12

page 13

page 16

page 18

page 19


Learning to Factor Policies and Action-Value Functions: Factored Action Space Representations for Deep Reinforcement learning

Deep Reinforcement Learning (DRL) methods have performed well in an incr...

Action Q-Transformer: Visual Explanation in Deep Reinforcement Learning with Encoder-Decoder Model using Action Query

The excellent performance of Transformer in supervised learning has led ...

An Actor-Critic-Attention Mechanism for Deep Reinforcement Learning in Multi-view Environments

In reinforcement learning algorithms, leveraging multiple views of the e...

Visual Explanation of Deep Q-Network for Robot Navigation by Fine-tuning Attention Branch

Robot navigation with deep reinforcement learning (RL) achieves higher p...

A Deep Reinforcement Learning Approach for Constrained Online Logistics Route Assignment

As online shopping prevails and e-commerce platforms emerge, there is a ...

ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages

In this paper, we introduce a novel method for enhancing the effectivene...

Ensemble Consensus-based Representation Deep Reinforcement Learning for Hybrid FSO/RF Communication Systems

Hybrid FSO/RF system requires an efficient FSO and RF link switching mec...

Please sign up or login with your details

Forgot password? Click here to reset