CACTUS: Detecting and Resolving Conflicts in Objective Functions

03/13/2021
by   Subhajit Das, et al.
18

Machine learning (ML) models are constructed by expert ML practitioners using various coding languages, in which they tune and select models hyperparameters and learning algorithms for a given problem domain. They also carefully design an objective function or loss function (often with multiple objectives) that captures the desired output for a given ML task such as classification, regression, etc. In multi-objective optimization, conflicting objectives and constraints is a major area of concern. In such problems, several competing objectives are seen for which no single optimal solution is found that satisfies all desired objectives simultaneously. In the past VA systems have allowed users to interactively construct objective functions for a classifier. In this paper, we extend this line of work by prototyping a technique to visualize multi-objective objective functions either defined in a Jupyter notebook or defined using an interactive visual interface to help users to: (1) perceive and interpret complex mathematical terms in it and (2) detect and resolve conflicting objectives. Visualization of the objective function enlightens potentially conflicting objectives that obstructs selecting correct solution(s) for the desired ML task or goal. We also present an enumeration of potential conflicts in objective specification in multi-objective objective functions for classifier selection. Furthermore, we demonstrate our approach in a VA system that helps users in specifying meaningful objective functions to a classifier by detecting and resolving conflicting objectives and constraints. Through a within-subject quantitative and qualitative user study, we present results showing that our technique helps users interactively specify meaningful objective functions by resolving potential conflicts for a classification task.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/19/2022

An Analysis of the Admissibility of the Objective Functions Applied in Evolutionary Multi-objective Clustering

A variety of clustering criteria has been applied as an objective functi...
research
01/24/2022

IMO^3: Interactive Multi-Objective Off-Policy Optimization

Most real-world optimization problems have multiple objectives. A system...
research
12/01/2018

Hierarchical Genetic Algorithms with evolving objective functions

We propose a framework of genetic algorithms which use multi-level hiera...
research
12/31/2018

Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)

Utility functions or their equivalents (value functions, objective funct...
research
12/10/2022

How to select an objective function using information theory

Science tests competing theories or models by evaluating the similarity ...
research
03/25/2022

Machine-Learning Based Objective Function Selection for Community Detection

NECTAR, a Node-centric ovErlapping Community deTection AlgoRithm, presen...
research
06/30/2017

Neural Sequence Model Training via α-divergence Minimization

We propose a new neural sequence model training method in which the obje...

Please sign up or login with your details

Forgot password? Click here to reset