Transparently Capturing Request Execution Path for Anomaly Detection

01/20/2020
by   Yong Yang, et al.
0

With the increasing scale and complexity of cloud systems and big data analytics platforms, it is becoming more and more challenging to understand and diagnose the processing of a service request in such distributed platforms. One way that helps to deal with this problem is to capture the complete end-to-end execution path of service requests among all involved components accurately. This paper presents REPTrace, a generic methodology for capturing such execution paths in a transparent fashion. We analyze a comprehensive list of execution scenarios, and propose principles and algorithms for generating the end-to-end request execution path for all the scenarios. Moreover, this paper presents an anomaly detection approach exploiting request execution paths to detect anomalies of the execution during request processing. The experiments on four popular distributed platforms with different workloads show that REPTrace can transparently capture the accurate request execution path with reasonable latency and negligible network overhead. Fault injection experiments show that execution anomalies are detected with high recall (96

READ FULL TEXT

page 1

page 2

page 3

page 4

09/18/2019

Anomaly Detection As-a-Service

Cloud systems are complex, large, and dynamic systems whose behavior mus...
07/30/2018

Call Detail Records Driven Anomaly Detection and Traffic Prediction in Mobile Cellular Networks

Mobile networks possess information about the users as well as the netwo...
06/28/2018

Detecting Port and Net Scan using Apache Spark

Today, due to the high number of attacks and of anomalous events in netw...
06/17/2019

Slicing the IO execution with ReLayTracer

Analyzing IO performance anomalies is a crucial task in various computin...
07/25/2020

Insightful Assistant: AI-compatible Operation Graph Representations for Enhancing Industrial Conversational Agents

Advances in voice-controlled assistants paved the way into the consumer ...
06/03/2020

Serving DNNs like Clockwork: Performance Predictability from the Bottom Up

Machine learning inference is becoming a core building block for interac...
04/13/2020

Detecting Latency Degradation Patterns in Service-based Systems

Performance in heterogeneous service-based systems shows non-determistic...