Threshy: Supporting Safe Usage of Intelligent Web Services

08/19/2020
by   Alex Cummaudo, et al.
0

Increased popularity of `intelligent' web services provides end-users with machine-learnt functionality at little effort to developers. However, these services require a decision threshold to be set which is dependent on problem-specific data. Developers lack a systematic approach for evaluating intelligent services and existing evaluation tools are predominantly targeted at data scientists for pre-development evaluation. This paper presents a workflow and supporting tool, Threshy, to help software developers select a decision threshold suited to their problem domain. Unlike existing tools, Threshy is designed to operate in multiple workflows including pre-development, pre-release, and support. Threshy is designed for tuning the confidence scores returned by intelligent web services and does not deal with hyper-parameter optimisation used in ML models. Additionally, it considers the financial impacts of false positives. Threshold configuration files exported by Threshy can be integrated into client applications and monitoring infrastructure. Demo: https://bit.ly/2YKeYhE.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

10/17/2019

Service Wrapper: a system for converting web data into web services

Web services are widely used in many areas via callable APIs, however, d...
05/27/2020

Beware the evolving 'intelligent' web service! An integration architecture tactic to guard AI-first components

Intelligent services provide the power of AI to developers via simple RE...
06/18/2019

Losing Confidence in Quality: Unspoken Evolution of Computer Vision Services

Recent advances in artificial intelligence (AI) and machine learning (ML...
06/04/2019

Distributed Web browsing: supporting frequent uses and opportunistic requirements

Nowadays, the development of Web applications supporting distributed use...
07/19/2020

Geno: A Developer Tool for Authoring Multimodal Interaction on Existing Web Applications

Supporting voice commands in applications presents significant benefits ...
02/05/2022

Using SEQUAL for Identifying Requirements to Ecore Editors

Software engineers who use Model-Driven Development may be using Ecore f...
11/04/2019

Adapting a Container Infrastructure for Autonomous Vehicle Development

In the field of Autonomous Vehicle (AV) development, having a robust yet...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.