Spatio-attentive Graphs for Human-Object Interaction Detection

12/11/2020
by   Frederic Z. Zhang, et al.
3

We address the problem of detecting human–object interactions in images using graphical neural networks. Our network constructs a bipartite graph of nodes representing detected humans and objects, wherein messages passed between the nodes encode relative spatial and appearance information. Unlike existing approaches that separate appearance and spatial features, our method fuses these two cues within a single graphical model allowing information conditioned on both modalities to influence the prediction of interactions with neighboring nodes. Through extensive experimentation we demonstrate the advantages of fusing relative spatial information with appearance features in the computation of adjacency structure, message passing and the ultimate refined graph features. On the popular HICO-DET benchmark dataset, our model outperforms state-of-the-art with an mAP of 27.18, a 10

READ FULL TEXT

page 1

page 6

page 7

page 8

research
03/11/2020

VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions

Comprehensive visual understanding requires detection frameworks that ca...
research
08/19/2021

Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition

For a given video-based Human-Object Interaction scene, modeling the spa...
research
04/05/2019

Detecting Human-Object Interactions via Functional Generalization

We present an approach for detecting human-object interactions (HOIs) in...
research
08/23/2018

Learning Human-Object Interactions by Graph Parsing Neural Networks

This paper addresses the task of detecting and recognizing human-object ...
research
12/17/2020

LIGHTEN: Learning Interactions with Graph and Hierarchical TEmporal Networks for HOI in videos

Analyzing the interactions between humans and objects from a video inclu...
research
10/20/2020

Contextual Heterogeneous Graph Network for Human-Object Interaction Detection

Human-object interaction(HOI) detection is an important task for underst...
research
11/06/2017

PersonRank: Detecting Important People in Images

Always, some individuals in images are more important/attractive than ot...

Please sign up or login with your details

Forgot password? Click here to reset