Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML

by   Shweta Narkar, et al.

Automated Machine Learning (AutoML) is a rapidly growing set of technologies that automate the model development pipeline by searching model space and generating candidate models. A critical, final step of AutoML is human selection of a final model from dozens of candidates. In current AutoML systems, selection is supported only by performance metrics. Prior work has shown that in practice, people evaluate ML models based on additional criteria, such as the way a model makes predictions. Comparison may happen at multiple levels, from types of errors, to feature importance, to how the model makes predictions of specific instances. We developed to support interactive model comparison for AutoML by integrating multiple Explainable AI (XAI) and visualization techniques. We conducted a user study in which we both evaluated the system and used it as a technology probe to understand how users perform model comparison in an AutoML system. We discuss design implications for utilizing XAI techniques for model comparison and supporting the unique needs of data scientists in comparing AutoML models.


page 1

page 2

page 3

page 4


explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

We propose a framework for interactive and explainable machine learning ...

Invisible Users: Uncovering End-Users' Requirements for Explainable AI via Explanation Forms and Goals

Non-technical end-users are silent and invisible users of the state-of-t...

What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversations

Recently we see a rising number of methods in the field of eXplainable A...

Leveraging Rationales to Improve Human Task Performance

Machine learning (ML) systems across many application areas are increasi...

Towards Semantically Enhanced Data Understanding

In the field of machine learning, data understanding is the practice of ...

FoundWright: A System to Help People Re-find Pages from Their Web-history

Re-finding information is an essential activity, however, it can be diff...

Please sign up or login with your details

Forgot password? Click here to reset