DeepAI AI Chat
Log In Sign Up

Robotic Lever Manipulation using Hindsight Experience Replay and Shapley Additive Explanations

by   Sindre Benjamin Remman, et al.

This paper deals with robotic lever control using Explainable Deep Reinforcement Learning. First, we train a policy by using the Deep Deterministic Policy Gradient algorithm and the Hindsight Experience Replay technique, where the goal is to control a robotic manipulator to manipulate a lever. This enables us both to use continuous states and actions and to learn with sparse rewards. Being able to learn from sparse rewards is especially desirable for Deep Reinforcement Learning because designing a reward function for complex tasks such as this is challenging. We first train in the PyBullet simulator, which accelerates the training procedure, but is not accurate on this task compared to the real-world environment. After completing the training in PyBullet, we further train in the Gazebo simulator, which runs more slowly than PyBullet, but is more accurate on this task. We then transfer the policy to the real-world environment, where it achieves comparable performance to the simulated environments for most episodes. To explain the decisions of the policy we use the SHAP method to create an explanation model based on the episodes done in the real-world environment. This gives us some results that agree with intuition, and some that do not. We also question whether the independence assumption made when approximating the SHAP values influences the accuracy of these values for a system such as this, where there are some correlations between the states.


page 1

page 4

page 7


Soft Hindsight Experience Replay

Efficient learning in the environment with sparse rewards is one of the ...

Robotic Arm Control and Task Training through Deep Reinforcement Learning

This paper proposes a detailed and extensive comparison of the Trust Reg...

Rearrangement with Nonprehensile Manipulation Using Deep Reinforcement Learning

Rearranging objects on a tabletop surface by means of nonprehensile mani...

Approximating a deep reinforcement learning docking agent using linear model trees

Deep reinforcement learning has led to numerous notable results in robot...

Actively Learning Costly Reward Functions for Reinforcement Learning

Transfer of recent advances in deep reinforcement learning to real-world...

Towards advanced robotic manipulation

Robotic manipulation and control has increased in importance in recent y...