Scaling ML Products At Startups: A Practitioner's Guide

by   Atul Dhingra, et al.

How do you scale a machine learning product at a startup? In particular, how do you serve a greater volume, velocity, and variety of queries cost-effectively? We break down costs into variable costs-the cost of serving the model and performant-and fixed costs-the cost of developing and training new models. We propose a framework for conceptualizing these costs, breaking them into finer categories, and limn ways to reduce costs. Lastly, since in our experience, the most expensive fixed cost of a machine learning system is the cost of identifying the root causes of failures and driving continuous improvement, we present a way to conceptualize the issues and share our methodology for the same.


page 1

page 2

page 3

page 4


Model Exploration with Cost-Aware Learning

We present an extension to active learning routines in which non-constan...

Joint Coreset Construction and Quantization for Distributed Machine Learning

Coresets are small, weighted summaries of larger datasets, aiming at pro...

System to Integrate Fairness Transparently: An Industry Approach

There have been significant research efforts to address the issue of uni...

MLProxy: SLA-Aware Reverse Proxy for Machine Learning Inference Serving on Serverless Computing Platforms

Serving machine learning inference workloads on the cloud is still a cha...

Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures

With the advent of ubiquitous deployment of smart devices and the Intern...

On Combining Machine Learning with Decision Making

We present a new application and covering number bound for the framework...

A Penny a Function: Towards Cost Transparent Cloud Programming

Understanding and managing monetary cost factors is crucial when develop...

Please sign up or login with your details

Forgot password? Click here to reset