Studying Logging Practice in Machine Learning-based Applications

by   Patrick Loic Foalem, et al.

Logging is a common practice in traditional software development. Several research works have been done to investigate the different characteristics of logging practices in traditional software systems (e.g., Android applications, JAVA applications, C/C++ applications). Nowadays, we are witnessing more and more development of Machine Learning-based applications (ML-based applications). Today, there are many popular libraries that facilitate and contribute to the development of such applications, among which we can mention: Pytorch, Tensorflow, Theano, MXNet, Scikit-Learn, Caffe, and Keras. Despite the popularity of ML, we don't have a clear understanding of logging practices in ML applications. In this paper, we aim to fill this knowledge gap and help ML practitioners understand the characteristics of logging in ML-based applications. In particular, we conduct an empirical study on 110 open-source ML-based applications. Through a quantitative analysis, we find that logging practice in ML-based applications is less pervasive than in traditional applications including Android, JAVA, and C/C++ applications. Furthermore, the majority of logging statements in ML-based applications are in info and warn levels, compared to traditional applications where info is the majority of logging statement in C/C++ application and debug, error levels constitute the majority of logging statement in Android application. We also perform a quantitative and qualitative analysis of a random sample of logging statements to understand where ML developers put most of logging statements and examine why and how they are using logging. These analyses led to the following observations: (i) ML developers put most of the logging statements in model training, and in non-ML components. (ii) Data and model management appear to be the main reason behind the introduction of logging statements in ML-based applications.


page 1

page 2

page 3

page 4


MLSMM: Machine Learning Security Maturity Model

Assessing the maturity of security practices during the development of M...

Preliminary Systematic Literature Review of Machine Learning System Development Process

Previous machine learning (ML) system development research suggests that...

The Sense of Logging in the Linux Kernel

Logging plays a crucial role in software engineering because it is key t...

Do Not Take It for Granted: Comparing Open-Source Libraries for Software Development Effort Estimation

In the past two decades, several Machine Learning (ML) libraries have be...

What Do Developers Ask About ML Libraries? A Large-scale Study Using Stack Overflow

Modern software systems are increasingly including machine learning (ML)...

What Causes Exceptions in Machine Learning Applications? Mining Machine Learning-Related Stack Traces on Stack Overflow

Machine learning (ML), including deep learning, has recently gained trem...
06/01/2019 Quantitative Overfitting Management for Human-in-the-loop ML Application Development

Simplifying machine learning (ML) application development, including dis...

Please sign up or login with your details

Forgot password? Click here to reset