Unbiased Scene Graph Generation from Biased Training

02/27/2020
by   Kaihua Tang, et al.
0

Today's scene graph generation (SGG) task is still far from practical, mainly due to the severe training bias, e.g., collapsing diverse human walk on/ sit on/lay on beach into human on beach. Given such SGG, the down-stream tasks such as VQA can hardly infer better scene structures than merely a bag of objects. However, debiasing in SGG is not trivial because traditional debiasing methods cannot distinguish between the good and bad bias, e.g., good context prior (e.g., person read book rather than eat) and bad long-tailed bias (e.g., behind/in front of collapsed to near). In this paper, we present a novel SGG framework based on causal inference but not the conventional likelihood. We first build a causal graph for SGG, and perform traditional biased training with the graph. Then, we propose to draw the counterfactual causality from the trained graph to infer the effect from the bad bias, which should be removed. In particular, we use Total Direct Effect as the proposed final predicate score for unbiased SGG. Note that our framework is agnostic to any SGG model and thus can be widely applied in the community who seeks unbiased predictions. By using the proposed Scene Graph Diagnosis toolkit on the SGG benchmark Visual Genome and several prevailing models, we observed significant improvements over the previous state-of-the-art methods.

READ FULL TEXT

page 2

page 3

page 8

page 16

page 17

research
01/18/2022

Resistance Training using Prior Bias: toward Unbiased Scene Graph Generation

Scene Graph Generation (SGG) aims to build a structured representation o...
research
07/11/2023

Unbiased Scene Graph Generation via Two-stage Causal Modeling

Despite the impressive performance of recent unbiased Scene Graph Genera...
research
03/17/2022

Biasing Like Human: A Cognitive Bias Framework for Scene Graph Generation

Scene graph generation is a sophisticated task because there is no speci...
research
09/28/2020

Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect

As the class size grows, maintaining a balanced dataset across many clas...
research
01/01/2023

Skew Class-balanced Re-weighting for Unbiased Scene Graph Generation

An unbiased scene graph generation (SGG) algorithm referred to as Skew C...
research
08/15/2022

CAME: Context-aware Mixture-of-Experts for Unbiased Scene Graph Generation

The scene graph generation has gained tremendous progress in recent year...
research
09/02/2020

PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation

Today, scene graph generation(SGG) task is largely limited in realistic ...

Please sign up or login with your details

Forgot password? Click here to reset