Malicious Network Traffic Detection via Deep Learning: An Information Theoretic View

09/16/2020
by   Erick Galinkin, et al.
5

The attention that deep learning has garnered from the academic community and industry continues to grow year over year, and it has been said that we are in a new golden age of artificial intelligence research. However, neural networks are still often seen as a "black box" where learning occurs but cannot be understood in a human-interpretable way. Since these machine learning systems are increasingly being adopted in security contexts, it is important to explore these interpretations. We consider an Android malware traffic dataset for approaching this problem. Then, using the information plane, we explore how homeomorphism affects learned representation of the data and the invariance of the mutual information captured by the parameters on that data. We empirically validate these results, using accuracy as a second measure of similarity of learned representations. Our results suggest that although the details of learned representations and the specific coordinate system defined over the manifold of all parameters differ slightly, the functional approximations are the same. Furthermore, our results show that since mutual information remains invariant under homeomorphism, only feature engineering methods that alter the entropy of the dataset will change the outcome of the neural network. This means that for some datasets and tasks, neural networks require meaningful, human-driven feature engineering or changes in architecture to provide enough information for the neural network to generate a sufficient statistic. Applying our results can serve to guide analysis methods for machine learning engineers and suggests that neural networks that can exploit the convolution theorem are equally accurate as standard convolutional neural networks, and can be more computationally efficient.

READ FULL TEXT

page 35

page 37

research
03/31/2022

Mutual information estimation for graph convolutional neural networks

Measuring model performance is a key issue for deep learning practitione...
research
01/24/2022

Analytic Mutual Information in Bayesian Neural Networks

Bayesian neural networks have successfully designed and optimized a robu...
research
06/01/2018

Inverting Supervised Representations with Autoregressive Neural Density Models

Understanding the nature of representations learned by supervised machin...
research
03/28/2021

Explaining Representation by Mutual Information

Science is used to discover the law of world. Machine learning can be us...
research
08/04/2020

Making Sense of CNNs: Interpreting Deep Representations Their Invariances with INNs

To tackle increasingly complex tasks, it has become an essential ability...
research
10/07/2019

Softmax Is Not an Artificial Trick: An Information-Theoretic View of Softmax in Neural Networks

Despite great popularity of applying softmax to map the non-normalised o...
research
05/13/2018

Doing the impossible: Why neural networks can be trained at all

As deep neural networks grow in size, from thousands to millions to bill...

Please sign up or login with your details

Forgot password? Click here to reset