Crowdsourcing via Annotator Co-occurrence Imputation and Provable Symmetric Nonnegative Matrix Factorization

by   Shahana Ibrahim, et al.

Unsupervised learning of the Dawid-Skene (D S) model from noisy, incomplete and crowdsourced annotations has been a long-standing challenge, and is a critical step towards reliably labeling massive data. A recent work takes a coupled nonnegative matrix factorization (CNMF) perspective, and shows appealing features: It ensures the identifiability of the D&S model and enjoys low sample complexity, as only the estimates of the co-occurrences of annotator labels are involved. However, the identifiability holds only when certain somewhat restrictive conditions are met in the context of crowdsourcing. Optimizing the CNMF criterion is also costly – and convergence assurances are elusive. This work recasts the pairwise co-occurrence based D S model learning problem as a symmetric NMF (SymNMF) problem – which offers enhanced identifiability relative to CNMF. In practice, the SymNMF model is often (largely) incomplete, due to the lack of co-labeled items by some annotators. Two lightweight algorithms are proposed for co-occurrence imputation. Then, a low-complexity shifted rectified linear unit (ReLU)-empowered SymNMF algorithm is proposed to identify the D S model. Various performance characterizations (e.g., missing co-occurrence recoverability, stability, and convergence) and evaluations are also presented.



There are no comments yet.


page 1

page 2

page 3

page 4


On Identifiability of Nonnegative Matrix Factorization

In this letter, we propose a new identification criterion that guarantee...

A Provably Correct and Robust Algorithm for Convolutive Nonnegative Matrix Factorization

In this paper, we propose a provably correct algorithm for convolutive n...

Crowdsourcing via Pairwise Co-occurrences: Identifiability and Algorithms

The data deluge comes with high demands for data labeling. Crowdsourcing...

Dropping Symmetry for Fast Symmetric Nonnegative Matrix Factorization

Symmetric nonnegative matrix factorization (NMF), a special but importan...

Matrix Factorization Equals Efficient Co-occurrence Representation

Matrix factorization is a simple and effective solution to the recommend...

Recovering Joint Probability of Discrete Random Variables from Pairwise Marginals

Learning the joint probability of random variables (RVs) lies at the hea...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.