DeepAI AI Chat
Log In Sign Up

Studying Logging Practice in Machine Learning-based Applications

01/10/2023
by   Patrick Loic Foalem, et al.
0

Logging is a common practice in traditional software development. Several research works have been done to investigate the different characteristics of logging practices in traditional software systems (e.g., Android applications, JAVA applications, C/C++ applications). Nowadays, we are witnessing more and more development of Machine Learning-based applications (ML-based applications). Today, there are many popular libraries that facilitate and contribute to the development of such applications, among which we can mention: Pytorch, Tensorflow, Theano, MXNet, Scikit-Learn, Caffe, and Keras. Despite the popularity of ML, we don't have a clear understanding of logging practices in ML applications. In this paper, we aim to fill this knowledge gap and help ML practitioners understand the characteristics of logging in ML-based applications. In particular, we conduct an empirical study on 110 open-source ML-based applications. Through a quantitative analysis, we find that logging practice in ML-based applications is less pervasive than in traditional applications including Android, JAVA, and C/C++ applications. Furthermore, the majority of logging statements in ML-based applications are in info and warn levels, compared to traditional applications where info is the majority of logging statement in C/C++ application and debug, error levels constitute the majority of logging statement in Android application. We also perform a quantitative and qualitative analysis of a random sample of logging statements to understand where ML developers put most of logging statements and examine why and how they are using logging. These analyses led to the following observations: (i) ML developers put most of the logging statements in model training, and in non-ML components. (ii) Data and model management appear to be the main reason behind the introduction of logging statements in ML-based applications.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/31/2021

Machine Learning Application Development: Practitioners' Insights

Nowadays, intelligent systems and services are getting increasingly popu...
10/12/2019

Preliminary Systematic Literature Review of Machine Learning System Development Process

Previous machine learning (ML) system development research suggests that...
08/13/2022

The Sense of Logging in the Linux Kernel

Logging plays a crucial role in software engineering because it is key t...
07/04/2022

Do Not Take It for Granted: Comparing Open-Source Libraries for Software Development Effort Estimation

In the past two decades, several Machine Learning (ML) libraries have be...
06/27/2019

What Do Developers Ask About ML Libraries? A Large-scale Study Using Stack Overflow

Modern software systems are increasingly including machine learning (ML)...
09/23/2022

A Preliminary Investigation of MLOps Practices in GitHub

Background. The rapid and growing popularity of machine learning (ML) ap...
06/01/2019

Quantitative Overfitting Management for Human-in-the-loop ML Application Development with ease.ml/meter

Simplifying machine learning (ML) application development, including dis...