Reproducible and Adaptable Log Data Generation for Sound Cybersecurity Experiments

11/15/2021
by   Rafael Uetz, et al.
0

Artifacts such as log data and network traffic are fundamental for cybersecurity research, e.g., in the area of intrusion detection. Yet, most research is based on artifacts that are not available to others or cannot be adapted to own purposes, thus making it difficult to reproduce and build on existing work. In this paper, we identify the challenges of artifact generation with the goal of conducting sound experiments that are valid, controlled, and reproducible. We argue that testbeds for artifact generation have to be designed specifically with reproducibility and adaptability in mind. To achieve this goal, we present SOCBED, our proof-of-concept implementation and the first testbed with a focus on generating realistic log data for cybersecurity experiments in a reproducible and adaptable manner. SOCBED enables researchers to reproduce testbed instances on commodity computers, adapt them according to own requirements, and verify their correct functionality. We evaluate SOCBED with an exemplary, practical experiment on detecting a multi-step intrusion of an enterprise network and show that the resulting experiment is indeed valid, controlled, and reproducible. Both SOCBED and the log dataset underlying our evaluation are freely available.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/16/2022

Maintainable Log Datasets for Evaluation of Intrusion Detection Systems

Intrusion detection systems (IDS) monitor system logs and network traffi...
research
02/24/2022

Experiments as Code: A Concept for Reproducible, Auditable, Debuggable, Reusable, Scalable Experiments

A common concern in experimental research is the auditability and reprod...
research
12/01/2012

An Evolution Strategy Approach toward Rule-set Generation for Network Intrusion Detection Systems (IDS)

With the increasing number of intrusions in system and network infrastru...
research
11/27/2018

A Real-Time Remote IDS Testbed for Connected Vehicles

Connected vehicles are becoming commonplace. A constant connection betwe...
research
07/13/2023

A Controlled Experiment on the Impact of Intrusion Detection False Alarm Rate on Analyst Performance

Organizations use intrusion detection systems (IDSes) to identify harmfu...
research
08/24/2023

Introducing a New Alert Data Set for Multi-Step Attack Analysis

Intrusion detection systems (IDS) reinforce cyber defense by autonomousl...
research
09/01/2022

Reproducibility Companion Paper: Describing Subjective Experiment Consistency by p-Value P-P Plot

In this paper we reproduce experimental results presented in our earlier...

Please sign up or login with your details

Forgot password? Click here to reset