Adoption and Effects of Software Engineering Best Practices in Machine Learning

07/28/2020
by   Alex Serban, et al.
0

The increasing reliance on applications with machine learning (ML) components calls for mature engineering techniques that ensure these are built in a robust and future-proof manner. We aim to empirically determine the state of the art in how teams develop, deploy and maintain software with ML components. We mined both academic and grey literature and identified 29 engineering best practices for ML applications. We conducted a survey among 313 practitioners to determine the degree of adoption for these practices and to validate their perceived effects. Using the survey responses, we quantified practice adoption, differentiated along demographic characteristics, such as geography or team size. We also tested correlations and investigated linear and non-linear relationships between practices and their perceived effect using various statistical models. Our findings indicate, for example, that larger teams tend to adopt more practices, and that traditional software engineering practices tend to have lower adoption than ML specific practices. Also, the statistical models can accurately predict perceived effects such as agility, software quality and traceability, from the degree of adoption for specific sets of practices. Combining practice adoption rates with practice importance, as revealed by statistical models, we identify practices that are important but have low adoption, as well as practices that are widely adopted but are less important for the effects we studied. Overall, our survey and the analysis of responses received provide a quantitative basis for assessment and step-wise improvement of practice adoption by ML teams.

READ FULL TEXT
research
03/01/2021

Practices for Engineering Trustworthy Machine Learning Applications

Following the recent surge in adoption of machine learning (ML), the neg...
research
08/11/2022

Achievement Unlocked: A Case Study on Gamifying DevOps Practices in Industry

Gamification is the use of game elements such as points, leaderboards, a...
research
11/17/2022

A Study of Adoption and Effects of DevOps Practices

Many organizations adopt DevOps practices and tools in order to break do...
research
03/16/2021

Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?

Following continuous software engineering practices, there has been an i...
research
10/29/2021

Diagnosing Web Data of ICTs to Provide Focused Assistance in Agricultural Adoptions

The past decade has witnessed a rapid increase in technology ownership a...
research
05/26/2021

An Empirical Study of Software Architecture for Machine Learning

Specific developmental and operational characteristics of machine learni...

Please sign up or login with your details

Forgot password? Click here to reset