Weak Labeling for Crowd Learning

04/26/2018
by   Iker Beñaran-Muñoz, et al.
0

Crowdsourcing has become very popular among the machine learning community as a way to obtain labels that allow a ground truth to be estimated for a given dataset. In most of the approaches that use crowdsourced labels, annotators are asked to provide, for each presented instance, a single class label. Such a request could be inefficient, that is, considering that the labelers may not be experts, that way to proceed could fail to take real advantage of the knowledge of the labelers. In this paper, the use of weak labeling for crowd learning is proposed, where the annotators may provide more than a single label per instance to try not to miss the real label. The main hypothesis is that, by allowing weak labeling, knowledge can be extracted from the labelers more efficiently by than in the standard crowd learning scenario. Empirical evidence which supports that hypothesis is presented.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/23/2018

Optimizing the Wisdom of the Crowd: Inference, Learning, and Teaching

The unprecedented demand for large amount of data has catalyzed the tren...
research
01/13/2013

Crowd Labeling: a survey

Recently, there has been a burst in the number of research projects on h...
research
09/20/2022

Modeling sequential annotations for sequence labeling with crowds

Crowd sequential annotations can be an efficient and cost-effective way ...
research
07/27/2022

Learning Hyper Label Model for Programmatic Weak Supervision

To reduce the human annotation efforts, the programmatic weak supervisio...
research
01/04/2023

Learning Ambiguity from Crowd Sequential Annotations

Most crowdsourcing learning methods treat disagreement between annotator...
research
05/07/2018

Label Refinery: Improving ImageNet Classification through Label Progression

Among the three main components (data, labels, and models) of any superv...
research
06/13/2021

Semi-verified Learning from the Crowd with Pairwise Comparisons

We study the problem of crowdsourced PAC learning of Boolean-valued func...

Please sign up or login with your details

Forgot password? Click here to reset