RGB Matters: Learning 7-DoF Grasp Poses on Monocular RGBD Images

by   Minghao Gou, et al.

General object grasping is an important yet unsolved problem in the field of robotics. Most of the current methods either generate grasp poses with few DoF that fail to cover most of the success grasps, or only take the unstable depth image or point cloud as input which may lead to poor results in some cases. In this paper, we propose RGBD-Grasp, a pipeline that solves this problem by decoupling 7-DoF grasp detection into two sub-tasks where RGB and depth information are processed separately. In the first stage, an encoder-decoder like convolutional neural network Angle-View Net(AVN) is proposed to predict the SO(3) orientation of the gripper at every location of the image. Consequently, a Fast Analytic Searching(FAS) module calculates the opening width and the distance of the gripper to the grasp point. By decoupling the grasp detection problem and introducing the stable RGB modality, our pipeline alleviates the requirement for the high-quality depth image and is robust to depth sensor noise. We achieve state-of-the-art results on GraspNet-1Billion dataset compared with several baselines. Real robot experiments on a UR5 robot with an Intel Realsense camera and a Robotiq two-finger gripper show high success rates for both single object scenes and cluttered scenes. Our code and trained model will be made publicly available.



There are no comments yet.


page 1

page 3

page 5

page 6


2.5D Image based Robotic Grasping

We consider the problem of robotic grasping using depth + RGB informatio...

GPR: Grasp Pose Refinement Network for Cluttered Scenes

Object grasping in cluttered scenes is a widely investigated field of ro...

S4G: Amodal Single-view Single-Shot SE(3) Grasp Detection in Cluttered Scenes

Grasping is among the most fundamental and long-lasting problems in robo...

6-DoF Contrastive Grasp Proposal Network

Proposing grasp poses for novel objects is an essential component for an...

A Novel Generative Convolutional Neural Network for Robot Grasp Detection on Gaussian Guidance

The vision-based grasp detection method is an important research directi...

On-Policy Pixel-Level Grasping Across the Gap Between Simulation and Reality

Grasp detection in cluttered scenes is a very challenging task for robot...

Generating Grasp Poses for a High-DOF Gripper Using Neural Networks

We present a learning-based method to represent grasp poses of a high-DO...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.