BigRoots: An Effective Approach for Root-cause Analysis of Stragglers in Big Data System

01/10/2018
by   Honggang Zhou, et al.
0

Stragglers are commonly believed to have a great impact on the performance of big data system. However, the reason to cause straggler is complicated. Previous works mostly focus on straggler detection, schedule level optimization and coarse-grained cause analysis. These methods cannot provide valuable insights to help users optimize their programs. In this paper, we propose BigRoots, a general method incorporating both framework and system features for root-cause analysis of stragglers in big data system. BigRoots considers features from big data framework such as shuffle read/write bytes and JVM garbage collection time, as well as system resource utilization such as CPU, I/O and network, which is able to detect both internal and external root causes of stragglers. We verify BigRoots by injecting high resource utilization across different system components and perform case studies to analyze different workloads in Hibench. The experimental results demonstrate that BigRoots is effective to identify the root cause of stragglers and provide useful guidance for performance optimization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/27/2018

A Frequency Scaling based Performance Indicator Framework for Big Data Systems

It is important for big data systems to identify their performance bottl...
research
12/26/2020

Toward Compact Data from Big Data

Bigdata is a dataset of which size is beyond the ability of handling a v...
research
01/30/2017

Survey on Models and Techniques for Root-Cause Analysis

Automation and computer intelligence to support complex human decisions ...
research
03/08/2021

Automatic Cause Detection of Performance Problems in Web Applications

The execution of similar units can be compared by their internal behavio...
research
05/13/2022

Automatic Root Cause Quantification for Missing Edges in JavaScript Call Graphs (Extended Version)

Building sound and precise static call graphs for real-world JavaScript ...
research
11/21/2017

HybridTune: Spatio-temporal Data and Model Driven Performance Diagnosis for Big Data Systems

With tremendous growing interests in Big Data systems, analyzing and fac...
research
04/06/2022

Data Justice Stories: A Repository of Case Studies

The idea of "data justice" is of recent academic vintage. It has arisen ...

Please sign up or login with your details

Forgot password? Click here to reset