Learning Grasp Affordance Reasoning through Semantic Relations

by   Paola Ardón, et al.

Reasoning about object affordances allows an autonomous agent to perform generalised manipulation tasks among object instances. While current approaches to grasp affordance estimation are effective, they are limited to a single hypothesis. We present an approach for detection and extraction of multiple grasp affordances on an object via visual input. We define semantics as a combination of multiple attributes, which yields benefits in terms of generalisation for grasp affordance prediction. We use Markov Logic Networks to build a knowledge base graph representation to obtain a probability distribution of grasp affordances for an object. To harvest the knowledge base, we collect and make available a novel dataset that relates different semantic attributes. We achieve reliable mappings of the predicted grasp affordances on the object by learning prototypical grasping patches from several examples. We show our method's generalisation capabilities on grasp affordance prediction for novel instances and compare with similar methods in the literature. Moreover, using a robotic platform, on simulated and real scenarios, we evaluate the success of the grasping task when conditioned on the grasp affordance prediction.


page 1

page 2

page 5

page 8


Knowledge-Augmented Dexterous Grasping with Incomplete Sensing

Humans can determine a proper strategy to grasp an object according to t...

Reasoning on Grasp-Action Affordances

Artificial intelligence is essential to succeed in challenging activitie...

Self-Assessment of Grasp Affordance Transfer

Reasoning about object grasp affordances allows an autonomous agent to e...

End-to-End Learning of Semantic Grasping

We consider the task of semantic robotic grasping, in which a robot pick...

GKNet: grasp keypoint network for grasp candidates detection

Contemporary grasp detection approaches employ deep learning to achieve ...

Distributed Bayesian Online Learning for Cooperative Manipulation

For tasks where the dynamics of multiple agents are physically coupled, ...

A novel integrated method of detection-grasping for specific object based on the box coordinate matching

To better care for the elderly and disabled, it is essential for service...

Please sign up or login with your details

Forgot password? Click here to reset