On Exploring Undetermined Relationships for Visual Relationship Detection

05/05/2019
by   Yibing Zhan, et al.
0

In visual relationship detection, human-notated relationships can be regarded as determinate relationships. However, there are still large amount of unlabeled data, such as object pairs with less significant relationships or even with no relationships. We refer to these unlabeled but potentially useful data as undetermined relationships. Although a vast body of literature exists, few methods exploit these undetermined relationships for visual relationship detection. In this paper, we explore the beneficial effect of undetermined relationships on visual relationship detection. We propose a novel multi-modal feature based undetermined relationship learning network (MF-URLN) and achieve great improvements in relationship detection. In detail, our MF-URLN automatically generates undetermined relationships by comparing object pairs with human-notated data according to a designed criterion. Then, the MF-URLN extracts and fuses features of object pairs from three complementary modals: visual, spatial, and linguistic modals. Further, the MF-URLN proposes two correlated subnetworks: one subnetwork decides the determinate confidence, and the other predicts the relationships. We evaluate the MF-URLN on two datasets: the Visual Relationship Detection (VRD) and the Visual Genome (VG) datasets. The experimental results compared with state-of-the-art methods verify the significant improvements made by the undetermined relationships, e.g., the top-50 relation detection recall improves from 19.5 dataset.

READ FULL TEXT

page 1

page 3

page 8

page 9

page 10

research
04/03/2019

Exploring the Semantics for Visual Relationship Detection

Scene graph construction / visual relationship detection from an image a...
research
03/20/2019

On Class Imbalance and Background Filtering in Visual Relationship Detection

In this paper we investigate the problems of class imbalance and irrelev...
research
05/10/2019

Relationship Detection Measures for Binary SoC Data

System-on-Chip (SoC) designs are used in every aspect of computing and t...
research
11/12/2021

Using Bayesian Network Analysis to Reveal Complex Natures of Relationships

Relationships are vital for mankind in many aspects. According to Maslow...
research
11/11/2022

Probabilistic Debiasing of Scene Graphs

The quality of scene graphs generated by the state-of-the-art (SOTA) mod...
research
10/27/2020

SIRI: Spatial Relation Induced Network For Spatial Description Resolution

Spatial Description Resolution, as a language-guided localization task, ...
research
11/02/2019

Visual Relationship Detection with Relative Location Mining

Visual relationship detection, as a challenging task used to find and di...

Please sign up or login with your details

Forgot password? Click here to reset