PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation

by   Shaotian Yan, et al.

Today, scene graph generation(SGG) task is largely limited in realistic scenarios, mainly due to the extremely long-tailed bias of predicate annotation distribution. Thus, tackling the class imbalance trouble of SGG is critical and challenging. In this paper, we first discover that when predicate labels have strong correlation with each other, prevalent re-balancing strategies(e.g., re-sampling and re-weighting) will give rise to either over-fitting the tail data(e.g., bench sitting on sidewalk rather than on), or still suffering the adverse effect from the original uneven distribution(e.g., aggregating varied parked on/standing on/sitting on into on). We argue the principal reason is that re-balancing strategies are sensitive to the frequencies of predicates yet blind to their relatedness, which may play a more important role to promote the learning of predicate features. Therefore, we propose a novel Predicate-Correlation Perception Learning(PCPL for short) scheme to adaptively seek out appropriate loss weights by directly perceiving and utilizing the correlation among predicate classes. Moreover, our PCPL framework is further equipped with a graph encoder module to better extract context features. Extensive experiments on the benchmark VG150 dataset show that the proposed PCPL performs markedly better on tail classes while well-preserving the performance on head ones, which significantly outperforms previous state-of-the-art methods.


page 1

page 4


Predicate correlation learning for scene graph generation

For a typical Scene Graph Generation (SGG) method, there is often a larg...

Dual-branch Hybrid Learning Network for Unbiased Scene Graph Generation

The current studies of Scene Graph Generation (SGG) focus on solving the...

Recovering the Unbiased Scene Graphs from the Biased Ones

Given input images, scene graph generation (SGG) aims to produce compreh...

BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition

Our work focuses on tackling the challenging but natural visual recognit...

Learning To Generate Scene Graph from Head to Tail

Scene Graph Generation (SGG) represents objects and their interactions w...

Unbiased Scene Graph Generation from Biased Training

Today's scene graph generation (SGG) task is still far from practical, m...

Domain Balancing: Face Recognition on Long-Tailed Domains

Long-tailed problem has been an important topic in face recognition task...