Learning a visuomotor controller for real world robotic grasping using simulated depth images

06/14/2017
by   Ulrich Viereck, et al.
0

We want to build robots that are useful in unstructured real world applications, such as doing work in the household. Grasping in particular is an important skill in this domain, yet it remains a challenge. One of the key hurdles is handling unexpected changes or motion in the objects being grasped and kinematic noise or other errors in the robot. This paper proposes an approach to learning a closed-loop controller for robotic grasping that dynamically guides the gripper to the object. We use a wrist-mounted sensor to acquire depth images in front of the gripper and train a convolutional neural network to learn a distance function to true grasps for grasp configurations over an image. The training sensor data is generated in simulation, a major advantage over previous work that uses real robot experience, which is costly to obtain. Despite being trained in simulation, our approach works well on real noisy sensor images. We compare our controller in simulated and real robot experiments to a strong baseline for grasp pose detection, and find that our approach significantly outperforms the baseline in the presence of kinematic noise, perceptual errors and disturbances of the object during grasping.

READ FULL TEXT

page 2

page 8

research
09/04/2019

Towards Precise Robotic Grasping by Probabilistic Post-grasp Displacement Estimation

Precise robotic grasping is important for many industrial applications, ...
research
10/10/2018

Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation

We develop an approach that benefits from large simulated datasets and t...
research
05/13/2020

Action Image Representation: Learning Scalable Deep Grasping Policies with Zero Real World Data

This paper introduces Action Image, a new grasp proposal representation ...
research
12/08/2017

Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control

In the context of deep learning for robotics, we show effective method o...
research
02/25/2019

A Versatile Framework for Robust and Adaptive Door Operation with a Mobile Manipulator Robot

The ability to deal with articulated objects is very important for robot...
research
02/16/2023

Learning a Meta-Controller for Dynamic Grasping

Grasping moving objects is a challenging task that combines multiple sub...
research
03/08/2023

Grasping Student: semi-supervised learning for robotic manipulation

Gathering real-world data from the robot quickly becomes a bottleneck wh...

Please sign up or login with your details

Forgot password? Click here to reset