Obstacle Avoidance for Robotic Manipulator in Joint Space via Improved Proximal Policy Optimization

10/03/2022
by   Yongliang Wang, et al.
0

Reaching tasks with random targets and obstacles can still be challenging when the robotic arm is operating in unstructured environments. In contrast to traditional model-based methods, model-free reinforcement learning methods do not require complex inverse kinematics or dynamics equations to be calculated. In this paper, we train a deep neural network via an improved Proximal Policy Optimization (PPO) algorithm, which aims to map from task space to joint space for a 6-DoF manipulator. In particular, we modify the original PPO and design an effective representation for environmental inputs and outputs to train the robot faster in a larger workspace. Firstly, a type of action ensemble is adopted to improve output efficiency. Secondly, the policy is designed to join in value function updates directly. Finally, the distance between obstacles and links of the manipulator is calculated based on a geometry method as part of the representation of states. Since training such a task in real-robot is time-consuming and strenuous, we develop a simulation environment to train the model. We choose Gazebo as our first simulation environment since it often produces a smaller Sim-to-Real gap than other simulators. However, the training process in Gazebo is time-consuming and takes a long time. Therefore, to address this limitation, we propose a Sim-to-Sim method to reduce the training time significantly. The trained model is finally used in a real-robot setup without fine-tuning. Experimental results showed that using our method, the robot was capable of tracking a single target or reaching multiple targets in unstructured environments.

READ FULL TEXT

page 1

page 2

page 3

page 5

research
11/12/2020

Joint Space Control via Deep Reinforcement Learning

The dominant way to control a robot manipulator uses hand-crafted differ...
research
05/17/2022

Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space

General-purpose robots require diverse repertoires of behaviors to compl...
research
10/31/2022

Reinforcement Learning for Solving Robotic Reaching Tasks in the Neurorobotics Platform

In recent years, reinforcement learning (RL) has shown great potential f...
research
09/11/2018

Danger-aware Weighted Advantage Composition of Deep Reinforcement Learning for Robot Navigation

Self-navigation, referring to automatically reaching the goal while avoi...
research
10/11/2018

A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies

Learning effective visuomotor policies for robots purely from data is ch...
research
09/11/2023

Task-Oriented Cross-System Design for Timely and Accurate Modeling in the Metaverse

In this paper, we establish a task-oriented cross-system design framewor...

Please sign up or login with your details

Forgot password? Click here to reset