Focusing and Calibration of Large Scale Network Sensors using GraphBLAS Anonymized Hypersparse Matrices

by   Jeremy Kepner, et al.

Defending community-owned cyber space requires community-based efforts. Large-scale network observations that uphold the highest regard for privacy are key to protecting our shared cyberspace. Deployment of the necessary network sensors requires careful sensor placement, focusing, and calibration with significant volumes of network observations. This paper demonstrates novel focusing and calibration procedures on a multi-billion packet dataset using high-performance GraphBLAS anonymized hypersparse matrices. The run-time performance on a real-world data set confirms previously observed real-time processing rates for high-bandwidth links while achieving significant data compression. The output of the analysis demonstrates the effectiveness of these procedures at focusing the traffic matrix and revealing the underlying stable heavy-tail statistical distributions that are necessary for anomaly detection. A simple model of the corresponding probability of detection (p_ d) and probability of false alarm (p_ fa) for these distributions highlights the criticality of network sensor focusing and calibration. Once a sensor is properly focused and calibrated it is then in a position to carry out two of the central tenets of good cybersecurity: (1) continuous observation of the network and (2) minimizing unbrokered network connections.


CalibNet: Self-Supervised Extrinsic Calibration using 3D Spatial Transformer Networks

3D LiDARs and 2D cameras are increasingly being used alongside each othe...

Large Scale Enrichment and Statistical Cyber Characterization of Network Traffic

Modern network sensors continuously produce enormous quantities of raw d...

CalibDNN: Multimodal Sensor Calibration for Perception Using Deep Neural Networks

Current perception systems often carry multimodal imagers and sensors su...

Deployment of Real-Time Network Traffic Analysis using GraphBLAS Hypersparse Matrices and D4M Associative Arrays

Matrix/array analysis of networks can provide significant insight into t...

GraphBLAS on the Edge: Anonymized High Performance Streaming of Network Traffic

Long range detection is a cornerstone of defense in many operating domai...

Please sign up or login with your details

Forgot password? Click here to reset