Robotic Lever Manipulation using Hindsight Experience Replay and Shapley Additive Explanations

10/07/2021
by   Sindre Benjamin Remman, et al.
0

This paper deals with robotic lever control using Explainable Deep Reinforcement Learning. First, we train a policy by using the Deep Deterministic Policy Gradient algorithm and the Hindsight Experience Replay technique, where the goal is to control a robotic manipulator to manipulate a lever. This enables us both to use continuous states and actions and to learn with sparse rewards. Being able to learn from sparse rewards is especially desirable for Deep Reinforcement Learning because designing a reward function for complex tasks such as this is challenging. We first train in the PyBullet simulator, which accelerates the training procedure, but is not accurate on this task compared to the real-world environment. After completing the training in PyBullet, we further train in the Gazebo simulator, which runs more slowly than PyBullet, but is more accurate on this task. We then transfer the policy to the real-world environment, where it achieves comparable performance to the simulated environments for most episodes. To explain the decisions of the policy we use the SHAP method to create an explanation model based on the episodes done in the real-world environment. This gives us some results that agree with intuition, and some that do not. We also question whether the independence assumption made when approximating the SHAP values influences the accuracy of these values for a system such as this, where there are some correlations between the states.

READ FULL TEXT

page 1

page 4

page 7

research
07/28/2021

Value-Based Reinforcement Learning for Continuous Control Robotic Manipulation in Multi-Task Sparse Reward Settings

Learning continuous control in high-dimensional sparse reward settings, ...
research
02/06/2020

Soft Hindsight Experience Replay

Efficient learning in the environment with sparse rewards is one of the ...
research
05/06/2020

Robotic Arm Control and Task Training through Deep Reinforcement Learning

This paper proposes a detailed and extensive comparison of the Trust Reg...
research
03/15/2018

Rearrangement with Nonprehensile Manipulation Using Deep Reinforcement Learning

Rearranging objects on a tabletop surface by means of nonprehensile mani...
research
11/23/2022

Actively Learning Costly Reward Functions for Reinforcement Learning

Transfer of recent advances in deep reinforcement learning to real-world...
research
09/19/2022

Towards advanced robotic manipulation

Robotic manipulation and control has increased in importance in recent y...
research
03/01/2022

Approximating a deep reinforcement learning docking agent using linear model trees

Deep reinforcement learning has led to numerous notable results in robot...

Please sign up or login with your details

Forgot password? Click here to reset