Threshy: Supporting Safe Usage of Intelligent Web Services

08/19/2020
by   Alex Cummaudo, et al.
0

Increased popularity of `intelligent' web services provides end-users with machine-learnt functionality at little effort to developers. However, these services require a decision threshold to be set which is dependent on problem-specific data. Developers lack a systematic approach for evaluating intelligent services and existing evaluation tools are predominantly targeted at data scientists for pre-development evaluation. This paper presents a workflow and supporting tool, Threshy, to help software developers select a decision threshold suited to their problem domain. Unlike existing tools, Threshy is designed to operate in multiple workflows including pre-development, pre-release, and support. Threshy is designed for tuning the confidence scores returned by intelligent web services and does not deal with hyper-parameter optimisation used in ML models. Additionally, it considers the financial impacts of false positives. Threshold configuration files exported by Threshy can be integrated into client applications and monitoring infrastructure. Demo: https://bit.ly/2YKeYhE.

READ FULL TEXT
research
10/17/2019

Service Wrapper: a system for converting web data into web services

Web services are widely used in many areas via callable APIs, however, d...
research
05/27/2020

Beware the evolving 'intelligent' web service! An integration architecture tactic to guard AI-first components

Intelligent services provide the power of AI to developers via simple RE...
research
06/18/2019

Losing Confidence in Quality: Unspoken Evolution of Computer Vision Services

Recent advances in artificial intelligence (AI) and machine learning (ML...
research
07/19/2020

Geno: A Developer Tool for Authoring Multimodal Interaction on Existing Web Applications

Supporting voice commands in applications presents significant benefits ...
research
12/15/2016

The CENDARI Infrastructure

The CENDARI infrastructure is a research supporting platform designed to...
research
05/13/2020

The Scalable Systems Laboratory: a Platform for Software Innovation for HEP

The Scalable Systems Laboratory (SSL), part of the IRIS-HEP Software Ins...
research
02/05/2022

Using SEQUAL for Identifying Requirements to Ecore Editors

Software engineers who use Model-Driven Development may be using Ecore f...

Please sign up or login with your details

Forgot password? Click here to reset