Exploring Network-Wide Flow Data with Flowyager

10/25/2020
by   Said Jawad Saidi, et al.
0

Many network operations, ranging from attack investigation and mitigation to traffic management, require answering network-wide flow queries in seconds. Although flow records are collected at each router, using available traffic capture utilities, querying the resulting datasets from hundreds of routers across sites and over time, remains a significant challenge due to the sheer traffic volume and distributed nature of flow records. In this paper, we investigate how to improve the response time for a priori unknown network-wide queries. We present Flowyager, a system that is built on top of existing traffic capture utilities. Flowyager generates and analyzes tree data structures, that we call Flowtrees, which are succinct summaries of the raw flow data available by capture utilities. Flowtrees are self-adjusted data structures that drastically reduce space and transfer requirements, by 75 to 95 transfers of Flowtrees, supports Flowtree operators, and provides a structured query language for answering flow queries across sites and time periods. By deploying a Flowyager prototype at both a large Internet Exchange Point and a Tier-1 Internet Service Provider, we showcase its capabilities for networks with hundreds of router interfaces. Our results show that the query response time can be reduced by an order of magnitude when compared with alternative data analytics platforms. Thus, Flowyager enables interactive network-wide queries and offers unprecedented drill-down capabilities to, e.g., identify DDoS culprits, pinpoint the involved sites, and determine the length of the attack.

READ FULL TEXT

page 5

page 13

page 16

research
05/25/2021

FENXI: Deep-learning Traffic Analytics at the Edge

Live traffic analysis at the first aggregation point in the ISP network ...
research
10/01/2018

Adaptive Planar Point Location

We present self-adjusting data structures for answering point location q...
research
03/12/2018

Reactive Proximity Data Structures for Graphs

We consider data structures for graphs where we maintain a subset of the...
research
12/28/2018

Do we have the time for IRM?: Service denial attacks and SDN-based defences

Distributed sensor networks such as IoT deployments generate large quant...
research
04/13/2023

Space-Time Tradeoffs for Conjunctive Queries with Access Patterns

In this paper, we investigate space-time tradeoffs for answering conjunc...
research
06/24/2020

Anycast Agility: Adaptive Routing to Manage DDoS

IP Anycast is used for services such as DNS and Content Delivery Network...
research
05/08/2023

On Blowback Traffic on the Internet

This paper considers the phenomenon where a single probe to a target gen...

Please sign up or login with your details

Forgot password? Click here to reset