Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure

by   Besmira Nushi, et al.

As machine learning systems move from computer-science laboratories into the open world, their accountability becomes a high priority problem. Accountability requires deep understanding of system behavior and its failures. Current evaluation methods such as single-score error metrics and confusion matrices provide aggregate views of system performance that hide important shortcomings. Understanding details about failures is important for identifying pathways for refinement, communicating the reliability of systems in different settings, and for specifying appropriate human oversight and engagement. Characterization of failures and shortcomings is particularly complex for systems composed of multiple machine learned components. For such systems, existing evaluation methods have limited expressiveness in describing and explaining the relationship among input content, the internal states of system components, and final output quality. We present Pandora, a set of hybrid human-machine methods and tools for describing and explaining system failures. Pandora leverages both human and system-generated observations to summarize conditions of system malfunction with respect to the input content and system architecture. We share results of a case study with a machine learning pipeline for image captioning that show how detailed performance views can be beneficial for analysis and debugging.


Compensating for Sensing Failures via Delegation in Human-AI Hybrid Systems

Given an increasing prevalence of intelligent systems capable of autonom...

Human Action Adverb Recognition: ADHA Dataset and A Three-Stream Hybrid Model

We introduce the first benchmark for a new problem --- recognizing human...

Overoptimization Failures and Specification Gaming in Multi-agent Systems

Overoptimization failures in machine learning and AI can involve specifi...

A Holistic Assessment of the Reliability of Machine Learning Systems

As machine learning (ML) systems increasingly permeate high-stakes setti...

The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures

Interactive AI systems increasingly employ a human-in-the-loop strategy....

Actionable and Interpretable Fault Localization for Recurring Failures in Online Service Systems

Fault localization is challenging in an online service system due to its...

MetricsVis: A Visual Analytics System for Evaluating Employee Performance in Public Safety Agencies

Evaluating employee performance in organizations with varying workloads ...

Please sign up or login with your details

Forgot password? Click here to reset