A Protocol for Intelligible Interaction Between Agents That Learn and Explain

01/04/2023
by   Ashwin Srinivasan, et al.
12

Recent engineering developments have seen the emergence of Machine Learning (ML) as a powerful form of data analysis with widespread applicability beyond its historical roots in the design of autonomous agents. However, relatively little attention has been paid to the interaction between people and ML systems. Recent developments on Explainable ML address this by providing visual and textual information on how the ML system arrived at a conclusion. In this paper we view the interaction between humans and ML systems within the broader context of interaction between agents capable of learning and explanation. Within this setting, we argue that it is more helpful to view the interaction as characterised by two-way intelligibility of information rather than once-off explanation of a prediction. We formulate two-way intelligibility as a property of a communication protocol. Development of the protocol is motivated by a set of `Intelligibility Axioms' for decision-support systems that use ML with a human-in-the-loop. The axioms are intended as sufficient criteria to claim that: (a) information provided by a human is intelligible to an ML system; and (b) information provided by an ML system is intelligible to a human. The axioms inform the design of a general synchronous interaction model between agents capable of learning and explanation. We identify conditions of compatibility between agents that result in bounded communication, and define Weak and Strong Two-Way Intelligibility between agents as properties of the communication protocol.

READ FULL TEXT
research
07/25/2020

Towards Game Design via Creative Machine Learning (GDCML)

In recent years, machine learning (ML) systems have been increasingly ap...
research
06/14/2021

Pitfalls of Explainable ML: An Industry Perspective

As machine learning (ML) systems take a more prominent and central role ...
research
08/10/2022

Capturing Dependencies within Machine Learning via a Formal Process Model

The development of Machine Learning (ML) models is more than just a spec...
research
03/05/2019

A Grounded Interaction Protocol for Explainable Artificial Intelligence

Explainable Artificial Intelligence (XAI) systems need to include an exp...
research
05/10/2022

Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory

Understanding how ML models work is a prerequisite for responsibly desig...
research
05/12/2023

ML-Based Teaching Systems: A Conceptual Framework

As the shortage of skilled workers continues to be a pressing issue, exa...
research
12/23/2020

AutonoML: Towards an Integrated Framework for Autonomous Machine Learning

Over the last decade, the long-running endeavour to automate high-level ...

Please sign up or login with your details

Forgot password? Click here to reset