Threshy: Supporting Safe Usage of Intelligent Web Services

by   Alex Cummaudo, et al.

Increased popularity of `intelligent' web services provides end-users with machine-learnt functionality at little effort to developers. However, these services require a decision threshold to be set which is dependent on problem-specific data. Developers lack a systematic approach for evaluating intelligent services and existing evaluation tools are predominantly targeted at data scientists for pre-development evaluation. This paper presents a workflow and supporting tool, Threshy, to help software developers select a decision threshold suited to their problem domain. Unlike existing tools, Threshy is designed to operate in multiple workflows including pre-development, pre-release, and support. Threshy is designed for tuning the confidence scores returned by intelligent web services and does not deal with hyper-parameter optimisation used in ML models. Additionally, it considers the financial impacts of false positives. Threshold configuration files exported by Threshy can be integrated into client applications and monitoring infrastructure. Demo:



There are no comments yet.


page 3


Service Wrapper: a system for converting web data into web services

Web services are widely used in many areas via callable APIs, however, d...

Beware the evolving 'intelligent' web service! An integration architecture tactic to guard AI-first components

Intelligent services provide the power of AI to developers via simple RE...

Losing Confidence in Quality: Unspoken Evolution of Computer Vision Services

Recent advances in artificial intelligence (AI) and machine learning (ML...

Distributed Web browsing: supporting frequent uses and opportunistic requirements

Nowadays, the development of Web applications supporting distributed use...

Geno: A Developer Tool for Authoring Multimodal Interaction on Existing Web Applications

Supporting voice commands in applications presents significant benefits ...

Using SEQUAL for Identifying Requirements to Ecore Editors

Software engineers who use Model-Driven Development may be using Ecore f...

Adapting a Container Infrastructure for Autonomous Vehicle Development

In the field of Autonomous Vehicle (AV) development, having a robust yet...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.