DeepAI AI Chat
Log In Sign Up

Situation Recognition with Graph Neural Networks

08/14/2017
by   Ruiyu Li, et al.
UNIVERSITY OF TORONTO
The Chinese University of Hong Kong
0

We address the problem of recognizing situations in images. Given an image, the task is to predict the most salient verb (action), and fill its semantic roles such as who is performing the action, what is the source and target of the action, etc. Different verbs have different roles (e.g. attacking has weapon), and each role can take on many possible values (nouns). We propose a model based on Graph Neural Networks that allows us to efficiently capture joint dependencies between roles using neural networks defined on a graph. Experiments with different graph connectivities show that our approach that propagates information between roles significantly outperforms existing work, as well as multiple baselines. We obtain roughly 3-5 work in predicting the full situation. We also provide a thorough qualitative analysis of our model and influence of different roles in the verbs.

READ FULL TEXT

page 1

page 2

page 4

page 6

page 7

page 11

page 12

page 22

10/05/2020

CopulaGNN: Towards Integrating Representational and Correlational Roles of Graphs in Graph Neural Networks

Graph-structured data are ubiquitous. However, graphs encode diverse typ...
12/02/2019

Continuous Graph Neural Networks

This paper builds the connection between graph neural networks and tradi...
10/04/2016

Revisiting Role Discovery in Networks: From Node to Edge Roles

Previous work in network analysis has focused on modeling the mixed-memb...
12/10/2021

Rethinking the Two-Stage Framework for Grounded Situation Recognition

Grounded Situation Recognition (GSR), i.e., recognizing the salient acti...
10/09/2021

Graph Neural Networks in Real-Time Fraud Detection with Lambda Architecture

Transaction checkout fraud detection is an essential risk control compon...
01/24/2022

A Method to Predict Semantic Relations on Artificial Intelligence Papers

Predicting the emergence of links in large evolving networks is a diffic...
07/19/2017

Measuring Thematic Fit with Distributional Feature Overlap

In this paper, we introduce a new distributional method for modeling pre...