Bayes-Optimal Effort Allocation in Crowdsourcing: Bounds and Index Policies

12/31/2015
by   Weici Hu, et al.
0

We consider effort allocation in crowdsourcing, where we wish to assign labeling tasks to imperfect homogeneous crowd workers to maximize overall accuracy in a continuous-time Bayesian setting, subject to budget and time constraints. The Bayes-optimal policy for this problem is the solution to a partially observable Markov decision process, but the curse of dimensionality renders the computation infeasible. Based on the Lagrangian Relaxation technique in Adelman & Mersereau (2008), we provide a computationally tractable instance-specific upper bound on the value of this Bayes-optimal policy, which can in turn be used to bound the optimality gap of any other sub-optimal policy. In an approach similar in spirit to the Whittle index for restless multiarmed bandits, we provide an index policy for effort allocation in crowdsourcing and demonstrate numerically that it outperforms other stateof- arts and performs close to optimal solution.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/12/2014

Statistical Decision Making for Optimal Budget Allocation in Crowd Labeling

In crowd labeling, a large amount of unlabeled data instances are outsou...
research
10/14/2022

Continuous-in-time Limit for Bayesian Bandits

This paper revisits the bandit problem in the Bayesian setting. The Baye...
research
04/17/2023

Designing Policies for Truth: Combating Misinformation with Transparency and Information Design

Misinformation has become a growing issue on online social platforms (OS...
research
01/30/2013

An Anytime Algorithm for Decision Making under Uncertainty

We present an anytime algorithm which computes policies for decision pro...
research
07/21/2020

Flow Sampling: Accurate and Load-balanced Sampling Policies

Software-defined networking simplifies network monitoring by means of pe...
research
11/06/2017

Sequential Multi-Class Labeling in Crowdsourcing

We consider a crowdsourcing platform where workers' responses to questio...
research
10/28/2017

Distributed Server Allocation for Content Delivery Networks

We propose a dynamic formulation of file-sharing networks in terms of an...

Please sign up or login with your details

Forgot password? Click here to reset