Feature Extraction and Feature Selection: Reducing Data Complexity with Apache Spark

12/11/2017
by   Dimitrios Sisiaridis, et al.
0

Feature extraction and feature selection are the first tasks in pre-processing of input logs in order to detect cyber security threats and attacks while utilizing machine learning. When it comes to the analysis of heterogeneous data derived from different sources, these tasks are found to be time-consuming and difficult to be managed efficiently. In this paper, we present an approach for handling feature extraction and feature selection for security analytics of heterogeneous data derived from different network sensors. The approach is implemented in Apache Spark, using its python API, named pyspark.

READ FULL TEXT

page 3

page 8

research
05/07/2019

Feature Selection and Feature Extraction in Pattern Analysis: A Literature Review

Pattern analysis often requires a pre-processing stage for extracting or...
research
01/30/2017

Click Through Rate Prediction for Contextual Advertisment Using Linear Regression

This research presents an innovative and unique way of solving the adver...
research
08/08/2022

DeepTLS: comprehensive and high-performance feature extraction for encrypted traffic

Feature extraction is critical for TLS traffic analysis using machine le...
research
03/06/2022

EEG-based processing and classification methodologies for Autism Spectrum Disorder: A Review

Autism Spectrum Disorder is a lifelong neurodevelopmental condition whic...
research
08/22/2019

Applications of Nature-Inspired Algorithms for Dimension Reduction: Enabling Efficient Data Analytics

In [1], we have explored the theoretical aspects of feature selection an...
research
04/06/2022

Standardized feature extraction from pairwise conflicts applied to the train rescheduling problem

We propose a train rescheduling algorithm which applies a standardized f...
research
07/13/2017

Automation of Feature Engineering for IoT Analytics

This paper presents an approach for automation of interpretable feature ...

Please sign up or login with your details

Forgot password? Click here to reset