DeepAI AI Chat
Log In Sign Up

Role of Intrinsic Motivation in User Interface Design to Enhance Worker Performance in Amazon MTurk

by   Pushyami Kaveti, et al.

Biologists and scientists have been tackling the problem of marine life monitoring and fish stock estimation for many years now. Efforts are now directed to move towards non-intrusive methods, by utilizing specially designed underwater robots to collect images of the marine population. Training machine learning algorithms on the images collected, we can now estimate the population. This in turn helps to impose regulations to control overfishing. To train these models, however, we need annotated images. Annotation of large sets of images collected over a decade is quite challenging. Hence, we resort to Amazon Mechanical Turk (MTurk), a crowdsourcing platform, for the image annotation task. Although it is fast to get work done in MTurk, the work obtained is often of poor quality. This work aims to understand the human factors in designing Human Intelligence Tasks (HITs), from the perspective of the Self-Determination Theory. Applying elements from the theory, we design an HIT to increase the competence and motivation of the workers. Within our experimental framework, we find that the new interface significantly improves the accuracy of worker performance.


page 3

page 4


Needle in a Haystack: An Analysis of Finding Qualified Workers on MTurk for Summarization

The acquisition of high-quality human annotations through crowdsourcing ...

Collecting Image Description Datasets using Crowdsourcing

We describe our two new datasets with images described by humans. Both t...

Exploring Effectiveness of Inter-Microtask Qualification Tests in Crowdsourcing

Qualification tests in crowdsourcing are often used to pre-filter worker...

Cross-Model Image Annotation Platform with Active Learning

We have seen significant leapfrog advancement in machine learning in rec...

Estimating Glycemic Impact of Cooking Recipes via Online Crowdsourcing and Machine Learning

Consumption of diets with low glycemic impact is highly recommended for ...

Underwater Image Enhancement Using Pre-trained Transformer

The goal of this work is to apply a denoising image transformer to remov...

Toward Safer Crowdsourced Content Moderation

Though detection systems have been developed to identify obscene content...