Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities

by   Doris Xin, et al.

Machine learning (ML) is now commonplace, powering data-driven applications in various organizations. Unlike the traditional perception of ML in research, ML production pipelines are complex, with many interlocking analytical components beyond training, whose sub-parts are often run multiple times on overlapping subsets of data. However, there is a lack of quantitative evidence regarding the lifespan, architecture, frequency, and complexity of these pipelines to understand how data management research can be used to make them more efficient, effective, robust, and reproducible. To that end, we analyze the provenance graphs of 3000 production ML pipelines at Google, comprising over 450,000 models trained, spanning a period of over four months, in an effort to understand the complexity and challenges underlying production ML. Our analysis reveals the characteristics, components, and topologies of typical industry-strength ML pipelines at various granularities. Along the way, we introduce a specialized data model for representing and reasoning about repeatedly run components in these ML pipelines, which we call model graphlets. We identify several rich opportunities for optimization, leveraging traditional data management ideas. We show how targeting even one of these opportunities, i.e., identifying and pruning wasted computation that does not translate to model deployment, can reduce wasted computation cost by 50 compromising the model deployment cadence.



page 4

page 6


Towards autonomic orchestration of machine learning pipelines in future networks

Machine learning (ML) techniques are being increasingly used in mobile n...

L'Apprentissage Automatique dans la planification et le contrôle de la production : un état de l'art

Proper Production Planning and Control (PPC) is capital to have an edge ...

Auto-Validate: Unsupervised Data Validation Using Data-Domain Patterns Inferred from Data Lakes

Complex data pipelines are increasingly common in diverse applications s...

Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines

Input pipelines, which ingest and transform input data, are an essential...

On Designing Data Models for Energy Feature Stores

The digitization of the energy infrastructure enables new, data driven, ...

End-to-end Robustness for Sensing-Reasoning Machine Learning Pipelines

As machine learning (ML) being applied to many mission-critical scenario...

A Tensor Compiler for Unified Machine Learning Prediction Serving

Machine Learning (ML) adoption in the enterprise requires simpler and mo...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.