Affordance Transfer Learning for Human-Object Interaction Detection

04/07/2021
by   Zhi Hou, et al.
1

Reasoning the human-object interactions (HOI) is essential for deeper scene understanding, while object affordances (or functionalities) are of great importance for human to discover unseen HOIs with novel objects. Inspired by this, we introduce an affordance transfer learning approach to jointly detect HOIs with novel objects and recognize affordances. Specifically, HOI representations can be decoupled into a combination of affordance and object representations, making it possible to compose novel interactions by combining affordance representations and novel object representations from additional images, i.e. transferring the affordance to novel objects. With the proposed affordance transfer learning, the model is also capable of inferring the affordances of novel objects from known affordance representations. The proposed method can thus be used to 1) improve the performance of HOI detection, especially for the HOIs with unseen objects; and 2) infer the affordances of novel objects. Experimental results on two datasets, HICO-DET and HOI-COCO (from V-COCO), demonstrate significant improvements over recent state-of-the-art methods for HOI detection and object affordance detection. Code is available at https://github.com/zhihou7/HOI-CL

READ FULL TEXT

page 1

page 3

page 8

page 10

page 12

03/15/2021

Detecting Human-Object Interaction via Fabricated Compositional Learning

Human-Object Interaction (HOI) detection, inferring the relationships be...
08/19/2021

Exploiting Scene Graphs for Human-Object Interaction Detection

Human-Object Interaction (HOI) detection is a fundamental visual task ai...
07/24/2020

Visual Compositional Learning for Human-Object Interaction Detection

Human-Object interaction (HOI) detection aims to localize and infer rela...
07/06/2022

Chairs Can be Stood on: Overcoming Object Bias in Human-Object Interaction Detection

Detecting Human-Object Interaction (HOI) in images is an important step ...
03/29/2022

OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction

Learning how humans manipulate objects requires machines to acquire know...
02/24/2022

Effective Actor-centric Human-object Interaction Detection

While Human-Object Interaction(HOI) Detection has achieved tremendous ad...
03/06/2019

Deep Transfer Learning for Multiple Class Novelty Detection

We propose a transfer learning-based solution for the problem of multipl...

Code Repositories