CrowDC: A Divide-and-Conquer Approach for Paired Comparisons in Crowdsourcing

02/23/2023

∙

by   Ming-Hung Wang, et al.

∙

0

∙

share

Ranking a set of samples based on subjectivity, such as the experience quality of streaming video or the happiness of images, has been a typical crowdsourcing task. Numerous studies have employed paired comparison analysis to solve challenges since it reduces the workload for participants by allowing them to select a single solution. Nonetheless, to thoroughly compare all target combinations, the number of tasks increases quadratically. This paper presents “CrowDC”, a divide-and-conquer algorithm for paired comparisons. Simulation results show that when ranking more than 100 items, CrowDC can reduce 40-50 the number of tasks while maintaining 90-95 approach.