Role of Intrinsic Motivation in User Interface Design to Enhance Worker Performance in Amazon MTurk

by   Pushyami Kaveti, et al.

Biologists and scientists have been tackling the problem of marine life monitoring and fish stock estimation for many years now. Efforts are now directed to move towards non-intrusive methods, by utilizing specially designed underwater robots to collect images of the marine population. Training machine learning algorithms on the images collected, we can now estimate the population. This in turn helps to impose regulations to control overfishing. To train these models, however, we need annotated images. Annotation of large sets of images collected over a decade is quite challenging. Hence, we resort to Amazon Mechanical Turk (MTurk), a crowdsourcing platform, for the image annotation task. Although it is fast to get work done in MTurk, the work obtained is often of poor quality. This work aims to understand the human factors in designing Human Intelligence Tasks (HITs), from the perspective of the Self-Determination Theory. Applying elements from the theory, we design an HIT to increase the competence and motivation of the workers. Within our experimental framework, we find that the new interface significantly improves the accuracy of worker performance.



There are no comments yet.


page 3

page 4


Rethinking Crowdsourcing Annotation: Partial Annotation with Salient Labels for Multi-Label Image Classification

Annotated images are required for both supervised model training and eva...

Collecting Image Description Datasets using Crowdsourcing

We describe our two new datasets with images described by humans. Both t...

Exploring Effectiveness of Inter-Microtask Qualification Tests in Crowdsourcing

Qualification tests in crowdsourcing are often used to pre-filter worker...

Cross-Model Image Annotation Platform with Active Learning

We have seen significant leapfrog advancement in machine learning in rec...

Estimating Glycemic Impact of Cooking Recipes via Online Crowdsourcing and Machine Learning

Consumption of diets with low glycemic impact is highly recommended for ...

Underwater Image Enhancement Using Pre-trained Transformer

The goal of this work is to apply a denoising image transformer to remov...

But Who Protects the Moderators? The Case of Crowdsourced Image Moderation

Though detection systems have been developed to identify obscene content...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.