Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labels

03/04/2022
by   Tao Pu, et al.
0

Training the multi-label image recognition models with partial labels, in which merely some labels are known while others are unknown for each image, is a considerably challenging and practical task. To address this task, current algorithms mainly depend on pre-training classification or similarity models to generate pseudo labels for the unknown labels. However, these algorithms depend on sufficient multi-label annotations to train the models, leading to poor performance especially with low known label proportion. In this work, we propose to blend category-specific representation across different images to transfer information of known labels to complement unknown labels, which can get rid of pre-training models and thus does not depend on sufficient annotations. To this end, we design a unified semantic-aware representation blending (SARB) framework that exploits instance-level and prototype-level semantic representation to complement unknown labels by two complementary modules: 1) an instance-level representation blending (ILRB) module blends the representations of the known labels in an image to the representations of the unknown labels in another image to complement these unknown labels. 2) a prototype-level representation blending (PLRB) module learns more stable representation prototypes for each category and blends the representation of unknown labels with the prototypes of corresponding labels to complement these labels. Extensive experiments on the MS-COCO, Visual Genome, Pascal VOC 2007 datasets show that the proposed SARB framework obtains superior performance over current leading competitors on all known label proportion settings, i.e., with the mAP improvement of 4.6 known label proportion is 10 https://github.com/HCPLab-SYSU/HCP-MLR-PL.

READ FULL TEXT

page 1

page 3

research
12/21/2021

Structured Semantic Transfer for Multi-Label Recognition with Partial Labels

Multi-label image recognition is a fundamental yet practical task becaus...
research
05/23/2022

Heterogeneous Semantic Transfer for Multi-label Recognition with Partial Labels

Multi-label image recognition with partial labels (MLR-PL), in which som...
research
11/15/2022

Category-Adaptive Label Discovery and Noise Rejection for Multi-label Image Recognition with Partial Positive Labels

As a promising solution of reducing annotation cost, training multi-labe...
research
08/20/2019

Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition

Recognizing multiple labels of images is a practical and challenging tas...
research
04/08/2022

Semantic Representation and Dependency Learning for Multi-Label Image Recognition

Recently many multi-label image recognition (MLR) works have made signif...
research
05/05/2023

Leaf Cultivar Identification via Prototype-enhanced Learning

Plant leaf identification is crucial for biodiversity protection and con...
research
10/21/2021

Multi-label Classification with Partial Annotations using Class-aware Selective Loss

Large-scale multi-label classification datasets are commonly, and perhap...

Please sign up or login with your details

Forgot password? Click here to reset