MBTree: Detecting Encryption RAT Communication Using Malicious Behavior Tree

06/17/2020
by   Cong Dong, et al.
0

A key challenge for cybersecurity defense is to detect the encryption Remote Control Trojan (RAT) communication traces. It is still an open research problem to detect encryption RAT preciously in different environments. Previous studies in this area either cannot handle the encrypted content or perform unstable in a different environment. To tackle both problems, we present MBTree, a novel host-level signature based approach for encryption RAT traffic detection. MBTree consists of a structure named MLTree and a similarity matching mechanism. The MLTree integrates multiple directed packet payload size sequences as a host signature. Furthermore, the matching mechanism compares two MLTree to decide if an alarm is triggered. Compared with previous related studies, MBTree (i) is more accurate to characterize different encryption RATs; (ii) has more robust performance when emerging new benign applications in the test environment; (iii) can automatically create signatures from malicious traffic without requiring human interaction. For evaluation, we collect traffic from multiple sources and reorganize them in a sophisticated manner. The experiment results demonstrate that our proposed method is more precise and robust, especially in the situation with new emerging applications.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/17/2022

Machine Learning for Encrypted Malicious Traffic Detection: Approaches, Datasets and Comparative Study

As people's demand for personal privacy and data security becomes a prio...
research
11/21/2017

Intrusion Detection and Ubiquitous Host to Host Encryption

Growing concern for individual privacy, driven by an increased public aw...
research
06/19/2019

Secure Handshake Mechanism for Autonomous Flying Agents Using Robust Cryptosystem

The autonomous flying agents in a Network-centric environment and brings...
research
08/05/2020

MORTON: Detection of Malicious Routines in Large-Scale DNS Traffic

In this paper, we present MORTON, a system that identifies compromised e...
research
10/15/2020

Fully Homomorphic Encryption via Affine Automorphisms

Homomorphic encryption is a method used in cryptopgraphy to create progr...
research
10/29/2018

Conditionals in Homomorphic Encryption and Machine Learning Applications

Homomorphic encryption aims at allowing computations on encrypted data w...
research
01/12/2021

Masking Host Identity on Internet: Encrypted TLS/SSL Handshake

Network middle-boxes often classify the traffic flows on the Internet to...

Please sign up or login with your details

Forgot password? Click here to reset