A Validation and Quality Assessment Method with Metamorphic Relations for Unsupervised Machine Learning Software

by   Zhiyi Zhang, et al.
Wuhan University

Unsupervised machine learning is a task of modeling the underlying structure of "unlabeled data". Since learning algorithms have been incorporated into many real-world applications, validating the implementations of those algorithms becomes much more important in the aim of software quality assurance. However, validating unsupervised machine learning programs is challenging because there lacks of priori knowledge. Along this line, in this paper, we present a metamorphic testing based method for validating and characterizing unsupervised machine learning programs, and conduct an empirical study on a real-world machine learning tool. The results demonstrate to what extent a program may fit to the profile of a specific scenario, which help end-users or software practitioners comprehend its performance in a vivid and light-weight way. And the experimental findings also reveal the gap between theory and implementation in a software artifact which could be easily ignored by people without much practical experience. In our method, metamorphic relations can serve as one type of quality measure, and one type of guidelines for selecting suitable programs.


page 15

page 16

page 18

page 19

page 20

page 23

page 26

page 27


Prioritization of Metamorphic Relations to reduce the cost of testing

An oracle is a mechanism to decide whether the outputs of the program fo...

Distortion and Faults in Machine Learning Software

Machine learning software, deep neural networks (DNN) software in partic...

ConPredictor: Concurrency Defect Prediction in Real-World Applications

Concurrent programs are difficult to test due to their inherent non-dete...

Quality issues in Machine Learning Software Systems

Context: An increasing demand is observed in various domains to employ M...

New Evolutionary Computation Models and their Applications to Machine Learning

Automatic Programming is one of the most important areas of computer sci...

Generating Examples From CLI Usage: Can Transformers Help?

Continuous evolution in modern software often causes documentation, tuto...

Please sign up or login with your details

Forgot password? Click here to reset