DeepAI AI Chat
Log In Sign Up

Interactive Model Cards: A Human-Centered Approach to Model Documentation

05/05/2022
by   Anamaria Crisan, et al.
Tableau
Salesforce
9

Deep learning models for natural language processing (NLP) are increasingly adopted and deployed by analysts without formal training in NLP or machine learning (ML). However, the documentation intended to convey the model's details and appropriate use is tailored primarily to individuals with ML or NLP expertise. To address this gap, we conduct a design inquiry into interactive model cards, which augment traditionally static model cards with affordances for exploring model documentation and interacting with the models themselves. Our investigation consists of an initial conceptual study with experts in ML, NLP, and AI Ethics, followed by a separate evaluative study with non-expert analysts who use ML models in their work. Using a semi-structured interview format coupled with a think-aloud protocol, we collected feedback from a total of 30 participants who engaged with different versions of standard and interactive model cards. Through a thematic analysis of the collected data, we identified several conceptual dimensions that summarize the strengths and limitations of standard and interactive model cards, including: stakeholders; design; guidance; understandability interpretability; sensemaking skepticism; and trust safety. Our findings demonstrate the importance of carefully considered design and interactivity for orienting and supporting non-expert analysts using deep learning models, along with a need for consideration of broader sociotechnical contexts and organizational dynamics. We have also identified design elements, such as language, visual cues, and warnings, among others, that support interactivity and make non-interactive content accessible. We summarize our findings as design guidelines and discuss their implications for a human-centered approach towards AI/ML documentation.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/23/2023

Addressing UX Practitioners' Challenges in Designing ML Applications: an Interactive Machine Learning Approach

UX practitioners face novel challenges when designing user interfaces fo...
03/02/2023

DeepLens: Interactive Out-of-distribution Data Detection in NLP Models

Machine Learning (ML) has been widely used in Natural Language Processin...
11/04/2022

NLP Inspired Training Mechanics For Modeling Transient Dynamics

In recent years, Machine learning (ML) techniques developed for Natural ...
03/18/2023

Requirement Formalisation using Natural Language Processing and Machine Learning: A Systematic Review

Improvement of software development methodologies attracts developers to...
11/29/2022

Democratizing Machine Learning for Interdisciplinary Scholars: Report on Organizing the NLP+CSS Online Tutorial Series

Many scientific fields – including biology, health, education, and the s...
02/08/2021

Enhancing Human-Machine Teaming for Medical Prognosis Through Neural Ordinary Differential Equations (NODEs)

Machine Learning (ML) has recently been demonstrated to rival expert-lev...
05/13/2022

Perspectives on Incorporating Expert Feedback into Model Updates

Machine learning (ML) practitioners are increasingly tasked with develop...