Towards Governing Agent's Efficacy: Action-Conditional β-VAE for Deep Transparent Reinforcement Learning

11/11/2018
by   John Yang, et al.
6

We tackle the blackbox issue of deep neural networks in the settings of reinforcement learning (RL) where neural agents learn towards maximizing reward gains in an uncontrollable way. Such learning approach is risky when the interacting environment includes an expanse of state space because it is then almost impossible to foresee all unwanted outcomes and penalize them with negative rewards beforehand. Unlike reverse analysis of learned neural features from previous works, our proposed method tackles the blackbox issue by encouraging an RL policy network to learn interpretable latent features through an implementation of a disentangled representation learning method. Toward this end, our method allows an RL agent to understand self-efficacy by distinguishing its influences from uncontrollable environmental factors, which closely resembles the way humans understand their scenes. Our experimental results show that the learned latent factors not only are interpretable, but also enable modeling the distribution of entire visited state space with a specific action condition. We have experimented that this characteristic of the proposed structure can lead to ex post facto governance for desired behaviors of RL agents.

READ FULL TEXT

page 4

page 5

page 6

page 7

research
10/11/2021

Learning Temporally-Consistent Representations for Data-Efficient Reinforcement Learning

Deep reinforcement learning (RL) agents that exist in high-dimensional s...
research
03/11/2021

Analyzing the Hidden Activations of Deep Policy Networks: Why Representation Matters

We analyze the hidden activations of neural network policies of deep rei...
research
09/12/2021

HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation

Discrete-continuous hybrid action space is a natural setting in many pra...
research
09/04/2023

Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learning

The black-box nature of deep reinforcement learning (RL) hinders them fr...
research
09/04/2019

Learning sparse representations in reinforcement learning

Reinforcement learning (RL) algorithms allow artificial agents to improv...
research
08/26/2022

Visual processing in context of reinforcement learning

Although deep reinforcement learning (RL) has recently enjoyed many succ...
research
08/17/2020

Playing Catan with Cross-dimensional Neural Network

Catan is a strategic board game having interesting properties, including...

Please sign up or login with your details

Forgot password? Click here to reset