Approximating a deep reinforcement learning docking agent using linear model trees

by   Vilde B. Gjærum, et al.

Deep reinforcement learning has led to numerous notable results in robotics. However, deep neural networks (DNNs) are unintuitive, which makes it difficult to understand their predictions and strongly limits their potential for real-world applications due to economic, safety, and assurance reasons. To remedy this problem, a number of explainable AI methods have been presented, such as SHAP and LIME, but these can be either be too costly to be used in real-time robotic applications or provide only local explanations. In this paper, the main contribution is the use of a linear model tree (LMT) to approximate a DNN policy, originally trained via proximal policy optimization(PPO), for an autonomous surface vehicle with five control inputs performing a docking operation. The two main benefits of the proposed approach are: a) LMTs are transparent which makes it possible to associate directly the outputs (control actions, in our case) with specific values of the input features, b) LMTs are computationally efficient and can provide information in real-time. In our simulations, the opaque DNN policy controls the vehicle and the LMT runs in parallel to provide explanations in the form of feature attributions. Our results indicate that LMTs can be a useful component within digital assurance frameworks for autonomous ships.


Explaining a Deep Reinforcement Learning Docking Agent Using Linear Model Trees with User Adapted Visualization

Deep neural networks (DNNs) can be useful within the marine robotics fie...

Controlling an Autonomous Vehicle with Deep Reinforcement Learning

We present a control approach for autonomous vehicles based on deep rein...

Robotic Lever Manipulation using Hindsight Experience Replay and Shapley Additive Explanations

This paper deals with robotic lever control using Explainable Deep Reinf...

Transparency and Explanation in Deep Reinforcement Learning Neural Networks

Autonomous AI systems will be entering human society in the near future ...

Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning

Modern power grids are experiencing grand challenges caused by the stoch...

Multi-Objective Autonomous Braking System using Naturalistic Dataset

A deep reinforcement learning based multi-objective autonomous braking s...

Designing Interpretable Approximations to Deep Reinforcement Learning with Soft Decision Trees

In an ever expanding set of research and application areas, deep neural ...

Please sign up or login with your details

Forgot password? Click here to reset