Prospect Theory Based Crowdsourcing for Classification in the Presence of Spammers

09/03/2019
by   Baocheng Geng, et al.
0

We consider the M-ary classification problem via crowdsourcing, where crowd workers respond to simple binary questions and the answers are aggregated via decision fusion. The workers have a reject option to skip answering a question when they do not have the expertise, or when the confidence of answering that question correctly is low. We further consider that there are spammers in the crowd who respond to the questions with random guesses. Under the payment mechanism that encourages the reject option, we study the behavior of honest workers and spammers, whose objectives are to maximize their monetary rewards. To accurately characterize human behavioral aspects, we employ prospect theory to model the rationality of the crowd workers, whose perception of costs and probabilities are distorted based on some value and weight functions, respectively. Moreover, we estimate the number of spammers and employ a weighted majority voting decision rule, where we assign an optimal weight for every worker to maximize the system performance. The probability of correct classification and asymptotic system performance are derived. We also provide simulation results to demonstrate the effectiveness of our approach.

READ FULL TEXT
research
10/26/2017

Optimal Crowdsourced Classification with a Reject Option in the Presence of Spammers

We explore the design of an effective crowdsourcing system for an M-ary ...
research
02/12/2018

Distinguishing Question Subjectivity from Difficulty for Improved Crowdsourcing

The questions in a crowdsourcing task typically exhibit varying degrees ...
research
09/11/2018

Reducing Uncertainty of Schema Matching via Crowdsourcing with Accuracy Rates

Schema matching is a central challenge for data integration systems. Ins...
research
09/01/2016

Crowdsourcing with Unsure Option

One of the fundamental problems in crowdsourcing is the trade-off betwee...
research
06/24/2019

Measuring the Expertise of Workers for Crowdsourcing Applications

Crowdsourcing platforms enable companies to propose tasks to a large cro...
research
10/17/2011

Budget-Optimal Task Allocation for Reliable Crowdsourcing Systems

Crowdsourcing systems, in which numerous tasks are electronically distri...
research
03/08/2023

Estimation of the qualification and behavior of a contributor and aggregation of his answers in a crowdsourcing context

Crowdsourcing is the outsourcing of tasks to a crowd of contributors on ...

Please sign up or login with your details

Forgot password? Click here to reset