DeepAI AI Chat
Log In Sign Up

DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration

by   Pengyuan Wang, et al.
Technische Universität München
Toyota Motor Europe

The ability to successfully grasp objects is crucial in robotics, as it enables several interactive downstream applications. To this end, most approaches either compute the full 6D pose for the object of interest or learn to predict a set of grasping points. While the former approaches do not scale well to multiple object instances or classes yet, the latter require large annotated datasets and are hampered by their poor generalization capabilities to new geometries. To overcome these shortcomings, we propose to teach a robot how to grasp an object with a simple and short human demonstration. Hence, our approach neither requires many annotated images nor is it restricted to a specific geometry. We first present a small sequence of RGB-D images displaying a human-object interaction. This sequence is then leveraged to build associated hand and object meshes that represent the depicted interaction. Subsequently, we complete missing parts of the reconstructed object shape and estimate the relative transformation between the reconstruction and the visible object in the scene. Finally, we transfer the a-priori knowledge from the relative pose between object and human hand with the estimate of the current object pose in the scene into necessary grasping instructions for the robot. Exhaustive evaluations with Toyota's Human Support Robot (HSR) in real and synthetic environments demonstrate the applicability of our proposed methodology and its advantage in comparison to previous approaches.


page 1

page 4

page 5

page 6


Learning Task-Oriented Grasping from Human Activity Datasets

We propose to leverage a real-world, human activity RGB datasets to teac...

HandNeRF: Learning to Reconstruct Hand-Object Interaction Scene from a Single RGB Image

This paper presents a method to learn hand-object interaction prior for ...

Multi-Finger Grasping Like Humans

Robots with multi-fingered grippers could perform advanced manipulation ...

DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video

Dexterous multi-fingered robotic hands have a formidable action space, y...

Learning 6-DoF Object Poses to Grasp Category-level Objects by Language Instructions

This paper studies the task of any objects grasping from the known categ...

Learning Descriptor of Constrained Task from Demonstration

Constrained objects, such as doors and drawers are often complex and sha...

Measuring a Robot Hand's Graspable Region using Power and Precision Grasps

The variety of robotic hand designs and actuation schemes makes it diffi...