Systematic Literature Review of Validation Methods for AI Systems

07/26/2021
by   Lalli Myllyaho, et al.
0

Context: Artificial intelligence (AI) has made its way into everyday activities, particularly through new techniques such as machine learning (ML). These techniques are implementable with little domain knowledge. This, combined with the difficulty of testing AI systems with traditional methods, has made system trustworthiness a pressing issue. Objective: This paper studies the methods used to validate practical AI systems reported in the literature. Our goal is to classify and describe the methods that are used in realistic settings to ensure the dependability of AI systems. Method: A systematic literature review resulted in 90 papers. Systems presented in the papers were analysed based on their domain, task, complexity, and applied validation methods. Results: The validation methods were synthesized into a taxonomy consisting of trial, simulation, model-centred validation, and expert opinion. Failure monitors, safety channels, redundancy, voting, and input and output restrictions are methods used to continuously validate the systems after deployment. Conclusions: Our results clarify existing strategies applied to validation. They form a basis for the synthesization, assessment, and refinement of AI system validation in research and guidelines for validating individual systems in practice. While various validation strategies have all been relatively widely applied, only few studies report on continuous validation. Keywords: artificial intelligence, machine learning, validation, testing, V V, systematic literature review.

READ FULL TEXT
research
04/04/2019

A Systematic Literature Review about the impact of Artificial Intelligence on Autonomous Vehicle Safety

Autonomous Vehicles (AV) are expected to bring considerable benefits to ...
research
12/06/2021

Application of Artificial Intelligence and Machine Learning in Libraries: A Systematic Review

As the concept and implementation of cutting-edge technologies like arti...
research
12/10/2019

Datamorphic Testing: A Methodology for Testing AI Applications

With the rapid growth of the applications of machine learning (ML) and o...
research
01/09/2023

Architecting Safer Autonomous Aviation Systems

The aviation literature gives relatively little guidance to practitioner...
research
12/20/2019

Data Validation Infrastructure for R

Checking data quality against domain knowledge is a common activity that...
research
09/22/2021

AI in Osteoporosis

In this chapter we explore and evaluate methods for trabecular bone char...

Please sign up or login with your details

Forgot password? Click here to reset