TrimTuner: Efficient Optimization of Machine Learning Jobs in the Cloud via Sub-Sampling

11/09/2020
by   Pedro Mendes, et al.
0

This work introduces TrimTuner, the first system for optimizing machine learning jobs in the cloud to exploit sub-sampling techniques to reduce the cost of the optimization process while keeping into account user-specified constraints. TrimTuner jointly optimizes the cloud and application-specific parameters and, unlike state of the art works for cloud optimization, eschews the need to train the model with the full training set every time a new configuration is sampled. Indeed, by leveraging sub-sampling techniques and data-sets that are up to 60x smaller than the original one, we show that TrimTuner can reduce the cost of the optimization process by up to 50x. Further, TrimTuner speeds-up the recommendation process by 65x with respect to state of the art techniques for hyper-parameter optimization that use sub-sampling techniques. The reasons for this improvement are twofold: i) a novel domain specific heuristic that reduces the number of configurations for which the acquisition function has to be evaluated; ii) the adoption of an ensemble of decision trees that enables boosting the speed of the recommendation process by one additional order of magnitude.

READ FULL TEXT
research
05/06/2019

Lynceus: Tuning and Provisioning Data Analytic Jobs on a Budget

Many enterprises need to run data analytic jobs on the cloud. Significan...
research
12/07/2020

SpotTune: Leveraging Transient Resources for Cost-efficient Hyper-parameter Tuning in the Public Cloud

Hyper-parameter tuning (HPT) is crucial for many machine learning (ML) a...
research
03/22/2019

Sampling Acquisition Functions for Batch Bayesian Optimization

This paper presents Acquisition Thompson Sampling (ATS), a novel algorit...
research
02/22/2021

Characterizing and Optimizing EDA Flows for the Cloud

Cloud computing accelerates design space exploration in logic synthesis,...
research
08/05/2021

HyperJump: Accelerating HyperBand via Risk Modelling

In the literature on hyper-parameter tuning, a number of recent solution...
research
03/10/2022

Efficient Runtime Profiling for Black-box Machine Learning Services on Sensor Streams

In highly distributed environments such as cloud, edge and fog computing...
research
04/16/2018

Chronos: A Unifying Optimization Framework for Speculative Execution of Deadline-critical MapReduce Jobs

Meeting desired application deadlines in cloud processing systems such a...

Please sign up or login with your details

Forgot password? Click here to reset