Transparently Capturing Request Execution Path for Anomaly Detection

by   Yong Yang, et al.

With the increasing scale and complexity of cloud systems and big data analytics platforms, it is becoming more and more challenging to understand and diagnose the processing of a service request in such distributed platforms. One way that helps to deal with this problem is to capture the complete end-to-end execution path of service requests among all involved components accurately. This paper presents REPTrace, a generic methodology for capturing such execution paths in a transparent fashion. We analyze a comprehensive list of execution scenarios, and propose principles and algorithms for generating the end-to-end request execution path for all the scenarios. Moreover, this paper presents an anomaly detection approach exploiting request execution paths to detect anomalies of the execution during request processing. The experiments on four popular distributed platforms with different workloads show that REPTrace can transparently capture the accurate request execution path with reasonable latency and negligible network overhead. Fault injection experiments show that execution anomalies are detected with high recall (96


page 1

page 2

page 3

page 4


Anomaly Detection As-a-Service

Cloud systems are complex, large, and dynamic systems whose behavior mus...

Call Detail Records Driven Anomaly Detection and Traffic Prediction in Mobile Cellular Networks

Mobile networks possess information about the users as well as the netwo...

Detecting Port and Net Scan using Apache Spark

Today, due to the high number of attacks and of anomalous events in netw...

Slicing the IO execution with ReLayTracer

Analyzing IO performance anomalies is a crucial task in various computin...

Insightful Assistant: AI-compatible Operation Graph Representations for Enhancing Industrial Conversational Agents

Advances in voice-controlled assistants paved the way into the consumer ...

Serving DNNs like Clockwork: Performance Predictability from the Bottom Up

Machine learning inference is becoming a core building block for interac...

Detecting Latency Degradation Patterns in Service-based Systems

Performance in heterogeneous service-based systems shows non-determistic...