Fast and Low-cost Search for Efficient Cloud Configurations for HPC Workloads

The use of cloud computational resources has become increasingly important for companies and researchers to access on-demand and at any moment high-performance resources. However, given the wide variety of virtual machine types, network configurations, number of instances, among others, finding the best configuration that reduces costs and resource waste while achieving acceptable performance is a hard task even for specialists. Thus, many approaches to find these good or optimal configurations for a given program have been proposed. Observing the performance of an application in some configuration takes time and money. Therefore, most of the approaches aim not only to find good solutions but also to reduce the search cost. One approach is the use of Bayesian Optimization to observe the least amount possible of configurations, reducing the search cost while still finding good solutions. Another approach is the use of a technique named Paramount Iteration to make performance assumptions of HPC workloads without entirely executing them (early-stopping), reducing the cost of making one observation, and making it feasible to grid search solutions. In this work, we show that both techniques can be used together to do fewer and low-cost observations. We show that such an approach can recommend Pareto-optimal solutions that are on average 1.68x better than Random Searching and with a 6-time cheaper search.


page 1

page 4

page 6


Scout: An Experienced Guide to Find the Best Cloud Configuration

Finding the right cloud configuration for workloads is an essential step...

Micky: A Cheaper Alternative for Selecting Cloud Instances

Most cloud computing optimizers explore and improve one workload at a ti...

Tuneful: An Online Significance-Aware Configuration Tuner for Big Data Analytics

Distributed analytics engines such as Spark are a common choice for proc...

Noise in the Clouds: Influence of Network Performance Variability on Application Scalability

Cloud computing represents an appealing opportunity for cost-effective d...

Cello: Efficient Computer Systems Optimization with Predictive Early Termination and Censored Regression

Sample-efficient machine learning (SEML) has been widely applied to find...

Best-by-Simulations: A Framework for Comparing Efficiency of Reconfigurable Multicore Architectures on Workloads with Deadlines

Energy consumption is a major concern in multicore systems. Perhaps the ...

Low-Level Augmented Bayesian Optimization for Finding the Best Cloud VM

With the advent of big data applications, which tends to have longer exe...

Please sign up or login with your details

Forgot password? Click here to reset