MonoGraspNet: 6-DoF Grasping with a Single RGB Image

09/26/2022
by   Guangyao Zhai, et al.
0

6-DoF robotic grasping is a long-lasting but unsolved problem. Recent methods utilize strong 3D networks to extract geometric grasping representations from depth sensors, demonstrating superior accuracy on common objects but perform unsatisfactorily on photometrically challenging objects, e.g., objects in transparent or reflective materials. The bottleneck lies in that the surface of these objects can not reflect back accurate depth due to the absorption or refraction of light. In this paper, in contrast to exploiting the inaccurate depth data, we propose the first RGB-only 6-DoF grasping pipeline called MonoGraspNet that utilizes stable 2D features to simultaneously handle arbitrary object grasping and overcome the problems induced by photometrically challenging objects. MonoGraspNet leverages keypoint heatmap and normal map to recover the 6-DoF grasping poses represented by our novel representation parameterized with 2D keypoints with corresponding depth, grasping direction, grasping width, and angle. Extensive experiments in real scenes demonstrate that our method can achieve competitive results in grasping common objects and surpass the depth-based competitor by a large margin in grasping photometrically challenging objects. To further stimulate robotic manipulation research, we additionally annotate and open-source a multi-view and multi-scene real-world grasping dataset, containing 120 objects of mixed photometric complexity with 20M accurate grasping labels.

READ FULL TEXT

page 1

page 3

page 4

page 5

research
02/17/2022

TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and Grasping

Transparent objects are common in our daily life and frequently handled ...
research
02/23/2018

Edge-Based Recognition of Novel Objects for Robotic Grasping

In this paper, we investigate the problem of grasping novel objects in u...
research
04/28/2020

Transferable Active Grasping and Real Embodied Dataset

Grasping in cluttered scenes is challenging for robot vision systems, as...
research
10/21/2021

Fuzzy-Depth Objects Grasping Based on FSG Algorithm and a Soft Robotic Hand

Autonomous grasping is an important factor for robots physically interac...
research
05/05/2023

Clothes Grasping and Unfolding Based on RGB-D Semantic Segmentation

Clothes grasping and unfolding is a core step in robotic-assisted dressi...
research
06/26/2018

Plenoptic Monte Carlo Object Localization for Robot Grasping under Layered Translucency

In order to fully function in human environments, robot perception will ...
research
10/21/2022

Neural Fields for Robotic Object Manipulation from a Single Image

We present a unified and compact representation for object rendering, 3D...

Please sign up or login with your details

Forgot password? Click here to reset