Human-in-the-Loop Design Cycles – A Process Framework that Integrates Design Sprints, Agile Processes, and Machine Learning with Humans

02/29/2020
by   Chaehan So, et al.
0

Demands on more transparency of the backbox nature of machine learning models have led to the recent rise of human-in-the-loop in machine learning, i.e. processes that integrate humans in the training and application of machine learning models. The present work argues that this process requirement does not represent an obstacle but an opportunity to optimize the design process. Hence, this work proposes a new process framework, Human-in-the-learning-loop (HILL) Design Cycles - a design process that integrates the structural elements of agile and design thinking process, and controls the training of a machine learning model by the human in the loop. The HILL Design Cycles process replaces the qualitative user testing by a quantitative psychometric measurement instrument for design perception. The generated user feedback serves to train a machine learning model and to instruct the subsequent design cycle along four design dimensions (novelty, energy, simplicity, tool). Mapping the four-dimensional user feedback into user stories and priorities, the design sprint thus transforms the user feedback directly into the implementation process. The human in the loop is a quality engineer who scrutinizes the collected user feedback to prevents invalid data to enter machine learning model training.

READ FULL TEXT
research
03/06/2021

Putting Humans in the Natural Language Processing Loop: A Survey

How can we design Natural Language Processing (NLP) systems that learn f...
research
04/21/2023

Tokenization Tractability for Human and Machine Learning Model: An Annotation Study

Is tractable tokenization for humans also tractable for machine learning...
research
12/03/2017

Formalizing Interruptible Algorithms for Human over-the-loop Analytics

Traditional data mining algorithms are exceptional at seeing patterns in...
research
08/25/2023

Human-in-the-loop online just-in-time software defect prediction

Online Just-In-Time Software Defect Prediction (O-JIT-SDP) uses an onlin...
research
08/02/2021

A Survey of Human-in-the-loop for Machine Learning

Human-in-the-loop aims to train an accurate prediction model with minimu...
research
03/06/2020

Learning the Designer's Preferences to Drive Evolution

This paper presents the Designer Preference Model, a data-driven solutio...
research
11/08/2021

Losses, Dissonances, and Distortions

In this paper I present a study in using the losses and gradients obtain...

Please sign up or login with your details

Forgot password? Click here to reset