Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task Correlation Information for Label Aggregation in Crowdsourcing

10/25/2020
by   Hanlu Wu, et al.
2

Crowdsourcing has attracted much attention for its convenience to collect labels from non-expert workers instead of experts. However, due to the high level of noise from the non-experts, an aggregation model that learns the true label by incorporating the source credibility is required. In this paper, we propose a novel framework based on graph neural networks for aggregating crowd labels. We construct a heterogeneous graph between workers and tasks and derive a new graph neural network to learn the representations of nodes and the true labels. Besides, we exploit the unknown latent interaction between the same type of nodes (workers or tasks) by adding a homogeneous attention layer in the graph neural networks. Experimental results on 13 real-world datasets show superior performance over state-of-the-art models.

READ FULL TEXT
research
05/17/2019

Graph Mining Meets Crowdsourcing: Extracting Experts for Answer Aggregation

Aggregating responses from crowd workers is a fundamental task in the pr...
research
11/01/2021

Robust Deep Learning from Crowds with Belief Propagation

Crowdsourcing systems enable us to collect noisy labels from crowd worke...
research
02/27/2020

Heterogeneous Graph Neural Networks for Malicious Account Detection

We present, GEM, the first heterogeneous graph neural network approach f...
research
10/21/2015

Time-Sensitive Bayesian Information Aggregation for Crowdsourcing Systems

Crowdsourcing systems commonly face the problem of aggregating multiple ...
research
07/19/2018

Label Aggregation via Finding Consensus Between Models

Label aggregation is an efficient and low cost way to make large dataset...
research
06/08/2019

Doubly Robust Crowdsourcing

Large-scale labeled datasets are the indispensable fuel that ignites the...
research
06/01/2021

Low Complexity Recruitment for Collaborative Mobile Crowdsourcing Using Graph Neural Networks

Collaborative Mobile crowdsourcing (CMCS) allows entities, e.g., local a...

Please sign up or login with your details

Forgot password? Click here to reset