Towards evaluating and eliciting high-quality documentation for intelligent systems

11/17/2020
by   David Piorkowski, et al.
0

A vital component of trust and transparency in intelligent systems built on machine learning and artificial intelligence is the development of clear, understandable documentation. However, such systems are notorious for their complexity and opaqueness making quality documentation a non-trivial task. Furthermore, little is known about what makes such documentation "good." In this paper, we propose and evaluate a set of quality dimensions to identify in what ways this type of documentation falls short. Then, using those dimensions, we evaluate three different approaches for eliciting intelligent system documentation. We show how the dimensions identify shortcomings in such documentation and posit how such dimensions can be use to further enable users to provide documentation that is suitable to a given persona or use case.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

07/20/2021

Roadmap of Designing Cognitive Metrics for Explainable Artificial Intelligence (XAI)

More recently, Explainable Artificial Intelligence (XAI) research has sh...
12/14/2017

Three IQs of AI Systems and their Testing Methods

The rapid development of artificial intelligence has brought the artific...
08/03/2020

Enhancing autonomy transparency: an option-centric rationale approach

While the advances in artificial intelligence and machine learning empow...
10/22/2018

Towards a context-dependent numerical data quality evaluation framework

This paper focuses on numeric data, with emphasis on distinct characteri...
08/31/2020

What is an intelligent system?

The concept of intelligent system has emerged in information technology ...
09/14/2020

Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations

This paper systematically derives design dimensions for the structured e...
10/11/2019

A generalized intelligent quality-based approach for fusing multi-source information

In this paper, we propose a generalized intelligent quality-based approa...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.