Data-Efficient Learning for Sim-to-Real Robotic Grasping using Deep Point Cloud Prediction Networks

by   Xinchen Yan, et al.

Training a deep network policy for robot manipulation is notoriously costly and time consuming as it depends on collecting a significant amount of real world data. To work well in the real world, the policy needs to see many instances of the task, including various object arrangements in the scene as well as variations in object geometry, texture, material, and environmental illumination. In this paper, we propose a method that learns to perform table-top instance grasping of a wide variety of objects while using no real world grasping data, outperforming the baseline using 2.5D shape by 10 cloud of object, and use that to train a domain-invariant grasping policy. We formulate the learning process as a two-step procedure: 1) Learning a domain-invariant 3D shape representation of objects from about 76K episodes in simulation and about 530 episodes in the real world, where each episode lasts less than a minute and 2) Learning a critic grasping policy in simulation only based on the 3D shape representation from step 1. Our real world data collection in step 1 is both cheaper and faster compared to existing approaches as it only requires taking multiple snapshots of the scene using a RGBD camera. Finally, the learned 3D representation is not specific to grasping, and can potentially be used in other interaction tasks.


page 1

page 4

page 6

page 7

page 8


Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation

Dexterous robotic hands have the capability to interact with a wide vari...

Real-time policy generation and its application to robot grasping

Real time applications such as robotic require real time actions based o...

Learning 6-DOF Grasping Interaction with Deep Geometry-aware 3D Representations

This paper focuses on the problem of learning 6-DOF grasping with a para...

CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation

Task-relevant grasping is critical for industrial assembly, where downst...

Grasping Student: semi-supervised learning for robotic manipulation

Gathering real-world data from the robot quickly becomes a bottleneck wh...

Hierarchical Policies for Cluttered-Scene Grasping with Latent Plans

6D grasping in cluttered scenes is a longstanding robotic manipulation p...

Unknown Object Segmentation through Domain Adaptation

The ability to segment unknown objects in cluttered scenes has a profoun...

Please sign up or login with your details

Forgot password? Click here to reset