On the Design of Strategic Task Recommendations for Sustainable Crowdsourcing-Based Content Moderation

06/04/2021
by   Sainath Sanga, et al.
0

Crowdsourcing-based content moderation is a platform that hosts content moderation tasks for crowd workers to review user submissions (e.g. text, images and videos) and make decisions regarding the admissibility of the posted content, along with a gamut of other tasks such as image labeling and speech-to-text conversion. In an attempt to reduce cognitive overload at the workers and improve system efficiency, these platforms offer personalized task recommendations according to the worker's preferences. However, the current state-of-the-art recommendation systems disregard the effects on worker's mental health, especially when they are repeatedly exposed to content moderation tasks with extreme content (e.g. violent images, hate-speech). In this paper, we propose a novel, strategic recommendation system for the crowdsourcing platform that recommends jobs based on worker's mental status. Specifically, this paper models interaction between the crowdsourcing platform's recommendation system (leader) and the worker (follower) as a Bayesian Stackelberg game where the type of the follower corresponds to the worker's cognitive atrophy rate and task preferences. We discuss how rewards and costs should be designed to steer the game towards desired outcomes in terms of maximizing the platform's productivity, while simultaneously improving the working conditions of crowd workers.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/06/2021

From Zero to The Hero: A Collaborative Market Aware Recommendation System for Crowd Workers

The success of software crowdsourcing depends on active and trustworthy ...
research
04/08/2021

Strategic Information Revelation in Crowdsourcing Systems Without Verification

We study a crowdsourcing problem where the platform aims to incentivize ...
research
07/27/2018

Task Recommendation in Crowdsourcing Based on Learning Preferences and Reliabilities

Workers participating in a crowdsourcing platform can have a wide range ...
research
09/03/2023

How Crowd Worker Factors Influence Subjective Annotations: A Study of Tagging Misogynistic Hate Speech in Tweets

Crowdsourced annotation is vital to both collecting labelled data to tra...
research
07/05/2021

Nobody of the Crowd: An Empirical Evaluation on Worker Clustering in Topcoder

Context: Software crowdsourcing platforms typically employ extrinsic rew...
research
04/22/2019

Multimedia Crowdsourcing with Bounded Rationality: A Cognitive Hierarchy Perspective

In multimedia crowdsourcing, the requester's quality requirements and re...
research
04/22/2019

Multimedia Crowdsourcing with Bounded Rationality: A Cognitive Hierarchy Prospective

In multimedia crowdsourcing, the requester's quality requirements and re...

Please sign up or login with your details

Forgot password? Click here to reset