Crowdsourcing the Perception of Machine Teaching

02/05/2020
by   Jonggi Hong, et al.
4

Teachable interfaces can empower end-users to attune machine learning systems to their idiosyncratic characteristics and environment by explicitly providing pertinent training examples. While facilitating control, their effectiveness can be hindered by the lack of expertise or misconceptions. We investigate how users may conceptualize, experience, and reflect on their engagement in machine teaching by deploying a mobile teachable testbed in Amazon Mechanical Turk. Using a performance-based payment scheme, Mechanical Turkers (N = 100) are called to train, test, and re-train a robust recognition model in real-time with a few snapshots taken in their environment. We find that participants incorporate diversity in their examples drawing from parallels to how humans recognize objects independent of size, viewpoint, location, and illumination. Many of their misconceptions relate to consistency and model capabilities for reasoning. With limited variation and edge cases in testing, the majority of them do not change strategies on a second training attempt.

READ FULL TEXT

page 2

page 7

page 8

page 9

page 10

page 11

page 12

page 13

research
11/18/2016

Analysis of a Design Pattern for Teaching with Features and Labels

We study the task of teaching a machine to classify objects using featur...
research
11/12/2014

Collecting Image Description Datasets using Crowdsourcing

We describe our two new datasets with images described by humans. Both t...
research
06/16/2023

Evaluating hardware differences for crowdsourcing and traditional recruiting methods

The most frequently used method to collect research data online is crowd...
research
08/16/2022

Blind Users Accessing Their Training Images in Teachable Object Recognizers

Iteration of training and evaluating a machine learning model is an impo...
research
05/31/2017

Descriptions of Objectives and Processes of Mechanical Learning

In [1], we introduced mechanical learning and proposed 2 approaches to m...
research
07/22/2021

A novel testbed for investigating the impact of teleoperator dynamics on perceived environment dynamics

Human-in-the-loop telerobotic systems (HiLTS) are robotic tools designed...
research
01/12/2021

The Hidden Cost of Using Amazon Mechanical Turk for Research

This work shares unexpected findings obtained from the use of the Amazon...

Please sign up or login with your details

Forgot password? Click here to reset