Exploring Outliers in Crowdsourced Ranking for QoE

by   Qianqian Xu, et al.
University of the Chinese Academy of Sciences
Peking University
Michigan State University
The Hong Kong University of Science and Technology

Outlier detection is a crucial part of robust evaluation for crowdsourceable assessment of Quality of Experience (QoE) and has attracted much attention in recent years. In this paper, we propose some simple and fast algorithms for outlier detection and robust QoE evaluation based on the nonconvex optimization principle. Several iterative procedures are designed with or without knowing the number of outliers in samples. Theoretical analysis is given to show that such procedures can reach statistically good estimates under mild conditions. Finally, experimental results with simulated and real-world crowdsourcing datasets show that the proposed algorithms could produce similar performance to Huber-LASSO approach in robust ranking, yet with nearly 8 or 90 times speed-up, without or with a prior knowledge on the sparsity size of outliers, respectively. Therefore the proposed methodology provides us a set of helpful tools for robust QoE evaluation with crowdsourcing data.


page 1

page 2

page 3

page 4


Robust Clustering Using Outlier-Sparsity Regularization

Notwithstanding the popularity of conventional clustering algorithms suc...

Outlier-Robust Tensor Low-Rank Representation for Data Clustering

Low-rank tensor analysis has received widespread attention with many pra...

Matrix optimization based Euclidean embedding with outliers

Euclidean embedding from noisy observations containing outlier errors is...

Robust Subjective Visual Property Prediction from Crowdsourced Pairwise Labels

The problem of estimating subjective visual properties from image and vi...

Deep Robust Subjective Visual Property Prediction in Crowdsourcing

The problem of estimating subjective visual properties (SVP) of images (...

Gaining Outlier Resistance with Progressive Quantiles: Fast Algorithms and Theoretical Studies

Outliers widely occur in big-data applications and may severely affect s...

Please sign up or login with your details

Forgot password? Click here to reset