HashFlow For Better Flow Record Collection

12/05/2018
by   Zongyi Zhao, et al.
0

Collecting flow records is a common practice of network operators and researchers for monitoring, diagnosing and understanding a network. Traditional tools like NetFlow face great challenges when both the speed and the complexity of the network traffic increase. To keep pace up, we propose HashFlow, a tool for more efficient and accurate collection and analysis of flow records. The central idea of HashFlow is to maintain accurate records for elephant flows, but summarized records for mice flows, by applying a novel collision resolution and record promotion strategy to hash tables. The performance bound can be analyzed with a probabilistic model, and with this strategy, HashFlow achieves a better utilization of space, and also more accurate flow records, without bringing extra complexity. We have implemented HashFlow, as well as several latest flow measurement algorithms such as FlowRadar, HashPipe and ElasticSketch, in a P4 software switch. Then we use traces from different operational networks to evaluate them. In these experiments, for various types of traffic analysis applications, HashFlow consistently demonstrates a clearly better performance against its state-of-the-art competitors. For example, using a small memory of 1 MB, HashFlow can accurately record around 55K flows, which is often 12.5 HashFlow achieves a relative error of around 11.6 of the best competitor is 42.9 out of 250K flows with a size estimation error of 5.6 73.7 merits of HashFlow come with almost no degradation of throughput.

READ FULL TEXT

page 1

page 9

research
06/24/2013

A State-Space Approach for Optimal Traffic Monitoring via Network Flow Sampling

The robustness and integrity of IP networks require efficient tools for ...
research
02/02/2021

Low-Rate Overuse Flow Tracer (LOFT): An Efficient and Scalable Algorithm for Detecting Overuse Flows

Current probabilistic flow-size monitoring can only detect heavy hitters...
research
05/08/2019

Locality-Sensitive Sketching for Resilient Network Flow Monitoring

Network monitoring is vital in modern clouds and data center networks fo...
research
04/21/2020

Faster and More Accurate Measurement through Additive-Error Counters

Counters are a fundamental building block for networking applications su...
research
12/14/2020

Sketch for traffic measurement: design, optimization, application and implementation

Network measurement probes the underlying network to support upper-level...
research
09/10/2018

Flow Length and Size Distributions in Campus Internet Traffic

Efficiency of numerous flow-oriented solutions proposed in the literatur...
research
04/12/2017

Persistent Spread Measurement for Big Network Data Based on Register Intersection

Persistent spread measurement is to count the number of distinct element...

Please sign up or login with your details

Forgot password? Click here to reset