Towards Mitigating the Problem of Insufficient and Ambiguous Supervision in Online Crowdsourcing Annotation

10/20/2022
by   Qian-Wei Wang, et al.
0

In real-world crowdsourcing annotation systems, due to differences in user knowledge and cultural backgrounds, as well as the high cost of acquiring annotation information, the supervision information we obtain might be insufficient and ambiguous. To mitigate the negative impacts, in this paper, we investigate a more general and broadly applicable learning problem, i.e. semi-supervised partial label learning, and propose a novel method based on pseudo-labeling and contrastive learning. Following the key inventing principle, our method facilitate the partial label disambiguation process with unlabeled data and at the same time assign reliable pseudo-labels to weakly supervised examples. Specifically, our method learns from the ambiguous labeling information via partial cross-entropy loss. Meanwhile, high-accuracy pseudo-labels are generated for both partial and unlabeled examples through confidence-based thresholding and contrastive learning is performed in a hybrid unsupervised and supervised manner for more discriminative representations, while its supervision increases curriculumly. The two main components systematically work as a whole and reciprocate each other. In experiments, our method consistently outperforms all comparing methods by a significant margin and set up the first state-of-the-art performance for semi-supervised partial label learning on image benchmarks.

READ FULL TEXT

page 1

page 5

research
03/20/2023

Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

Semi-supervised learning (SSL) has attracted enormous attention due to i...
research
12/13/2022

Boosting Semi-Supervised Learning with Contrastive Complementary Labeling

Semi-supervised learning (SSL) has achieved great success in leveraging ...
research
11/24/2022

Learning with Partial Labels from Semi-supervised Perspective

Partial Label (PL) learning refers to the task of learning from the part...
research
06/10/2019

A cost-reducing partial labeling estimator in text classification problem

We propose a new approach to address the text classification problems wh...
research
03/17/2020

The Value of Nullspace Tuning Using Partial Label Information

In semi-supervised learning, information from unlabeled examples is used...
research
12/20/2020

Bayesian Semi-supervised Crowdsourcing

Crowdsourcing has emerged as a powerful paradigm for efficiently labelin...

Please sign up or login with your details

Forgot password? Click here to reset