Affordance Transfer Learning for Human-Object Interaction Detection

04/07/2021
by   Zhi Hou, et al.
1

Reasoning the human-object interactions (HOI) is essential for deeper scene understanding, while object affordances (or functionalities) are of great importance for human to discover unseen HOIs with novel objects. Inspired by this, we introduce an affordance transfer learning approach to jointly detect HOIs with novel objects and recognize affordances. Specifically, HOI representations can be decoupled into a combination of affordance and object representations, making it possible to compose novel interactions by combining affordance representations and novel object representations from additional images, i.e. transferring the affordance to novel objects. With the proposed affordance transfer learning, the model is also capable of inferring the affordances of novel objects from known affordance representations. The proposed method can thus be used to 1) improve the performance of HOI detection, especially for the HOIs with unseen objects; and 2) infer the affordances of novel objects. Experimental results on two datasets, HICO-DET and HOI-COCO (from V-COCO), demonstrate significant improvements over recent state-of-the-art methods for HOI detection and object affordance detection. Code is available at https://github.com/zhihou7/HOI-CL

READ FULL TEXT

page 1

page 3

page 8

page 10

page 12

research
03/15/2021

Detecting Human-Object Interaction via Fabricated Compositional Learning

Human-Object Interaction (HOI) detection, inferring the relationships be...
research
08/19/2021

Exploiting Scene Graphs for Human-Object Interaction Detection

Human-Object Interaction (HOI) detection is a fundamental visual task ai...
research
06/01/2023

Object pop-up: Can we infer 3D objects and their poses from human interactions alone?

The intimate entanglement between objects affordances and human poses is...
research
01/25/2021

Transferable Interactiveness Knowledge for Human-Object Interaction Detection

Human-Object Interaction (HOI) detection is an important problem to unde...
research
07/06/2022

Chairs Can be Stood on: Overcoming Object Bias in Human-Object Interaction Detection

Detecting Human-Object Interaction (HOI) in images is an important step ...
research
03/29/2022

OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction

Learning how humans manipulate objects requires machines to acquire know...
research
01/13/2020

Classifying All Interacting Pairs in a Single Shot

In this paper, we introduce a novel human interaction detection approach...

Please sign up or login with your details

Forgot password? Click here to reset