On the Scalability of Big Data Cyber Security Analytics Systems

by   Faheem Ullah, et al.

Big Data Cyber Security Analytics (BDCA) systems use big data technologies (e.g., Apache Spark) to collect, store, and analyze a large volume of security event data for detecting cyber-attacks. The volume of digital data in general and security event data in specific is increasing exponentially. The velocity with which the security event data is generated and fed into a BDCA system is unpredictable. Therefore, a BDCA system should be highly scalable to deal with the unpredictable increase/decrease in the velocity of security event data. However, there has been little effort to investigate the scalability of BDCA systems to identify and exploit the sources of scalability improvement. In this paper, we first investigate the scalability of a Spark-based BDCA system with default Spark settings. we then identify Spark configuration parameters (e.g., execution memory) that can significantly impact the scalability of a BDCA system. Based on the identified parameters, we finally propose a parameter-driven adaptation approach, SCALER, for optimizing a system's scalability. We have conducted a set of experiments by implementing a Spark-based BDCA system on a large-scale OpenStack cluster. We ran our experiments with four security datasets. We have found that (i) a BDCA system with default Spark configuration parameters deviates from ideal scalability by 59.5 impact scalability (iii) SCALER improves the BDCA system's scalability by 20.8 compared to the scalability with default Spark parameter setting. The findings of our study highlight the importance of exploring the parameter space of the underlying big data framework (e.g., Apache Spark) for scalable cyber security analytics.


page 1

page 2

page 3

page 4


Automated Big Traffic Analytics for Cyber Security

Network traffic analytics technology is a cornerstone for cyber security...

A Survey on Big Data for Network Traffic Monitoring and Analysis

Network Traffic Monitoring and Analysis (NTMA) represents a key componen...

Simulation-Based Cyber Data Collection Efficacy

Building upon previous research in honeynets and simulations, we present...

Architectural Tactics for Big Data Cybersecurity Analytic Systems: A Review

Context: Big Data Cybersecurity Analytics is aimed at protecting network...

A big data intelligence marketplace and secure analytics experimentation platform for the aviation industry

The unprecedented volume, diversity and richness of aviation data that c...

Learning-based Automatic Parameter Tuning for Big Data Analytics Frameworks

Big data analytics frameworks (BDAFs) have been widely used for data pro...

AMIC: An Adaptive Information Theoretic Method to Identify Multi-Scale Temporal Correlations in Big Time Series Data

Recent development in computing, sensing and crowd-sourced data have res...

Please sign up or login with your details

Forgot password? Click here to reset