Deep Reasoning with Knowledge Graph for Social Relationship Understanding

by   Zhouxia Wang, et al.

Social relationships (e.g., friends, couple etc.) form the basis of the social network in our daily life. Automatically interpreting such relationships bears a great potential for the intelligent systems to understand human behavior in depth and to better interact with people at a social level. Human beings interpret the social relationships within a group not only based on the people alone, and the interplay between such social relationships and the contextual information around the people also plays a significant role. However, these additional cues are largely overlooked by the previous studies. We found that the interplay between these two factors can be effectively modeled by a novel structured knowledge graph with proper message propagation and attention. And this structured knowledge can be efficiently integrated into the deep neural network architecture to promote social relationship understanding by an end-to-end trainable Graph Reasoning Model (GRM), in which a propagation mechanism is learned to propagate node message through the graph to explore the interaction between persons of interest and the contextual objects. Meanwhile, a graph attentional mechanism is introduced to explicitly reason about the discriminative objects to promote recognition. Extensive experiments on the public benchmarks demonstrate the superiority of our method over the existing leading competitors.


page 1

page 3

page 5

page 7


An End-to-End Network for Generating Social Relationship Graphs

Socially-intelligent agents are of growing interest in artificial intell...

Dual-Glance Model for Deciphering Social Relationships

Since the beginning of early civilizations, social relationships derived...

Visual Social Relationship Recognition

Social relationships form the basis of social structure of humans. Devel...

Knowledge-Embedded Routing Network for Scene Graph Generation

To understand a scene in depth not only involves locating/recognizing in...

"where is this relationship going?": Understanding Relationship Trajectories in Narrative Text

We examine a new commonsense reasoning task: given a narrative describin...

Recursive Social Behavior Graph for Trajectory Prediction

Social interaction is an important topic in human trajectory prediction ...

HLVU : A New Challenge to Test Deep Understanding of Movies the Way Humans do

In this paper we propose a new evaluation challenge and direction in the...

Please sign up or login with your details

Forgot password? Click here to reset