DeepAI
Log In Sign Up

A Ransomware Triage Approach using a Task Memory based on Meta-Transfer Learning Framework

07/21/2022
by   Jinting Zhu, et al.
0

Solutions for rapid prioritization of different ransomware have been raised to formulate fast response plans to minimize socioeconomic damage from the massive growth of ransomware attacks in recent years. To address this concern, we propose a ransomware triage approach that can rapidly classify and prioritize different ransomware classes. Our Siamese Neural Network (SNN) based approach utilizes a pre-trained ResNet18 network in a meta-learning fashion to reduce the biases in weight and parameter calculations typically associated with a machine learning model trained with a limited number of training samples. Instead of image features typically used as inputs to many existing machine learning-based triage applications, our approach uses the entropy features directly obtained from the ransomware binary files to improve feature representation, resilient to obfuscation noise, and computationally less expensive. Our triage approach can classify ransomware samples into the correct classes if the ransomware features significantly match known ransomware profiles. Our evaluation shows that this classification part of our proposed approach achieves the accuracy exceeding 88 classification only machine learning-based approaches. In addition, we offer a new triage strategy based on the normalized and regularized weight ratios that evaluate the level of similarity matching across ransomware classes to identify any risky and unknown ransomware (e.g., zero-day attacks) so that a rapid further analysis can be conducted

READ FULL TEXT VIEW PDF
12/01/2021

A Few-Shot Meta-Learning based Siamese Neural Network using Entropy Features for Ransomware Classification

Ransomware defense solutions that can quickly detect and classify differ...
10/26/2021

Task-Aware Meta Learning-based Siamese Neural Network for Classifying Obfuscated Malware

Malware authors apply different obfuscation techniques on the generic fe...
10/31/2019

RankML: a Meta Learning-Based Approach for Pre-Ranking Machine Learning Pipelines

The explosion of digital data has created multiple opportunities for org...
09/23/2020

Fair Meta-Learning For Few-Shot Classification

Artificial intelligence nowadays plays an increasingly prominent role in...
06/23/2018

Dynamic Spectrum Matching with One-shot Learning

Convolutional neural networks (CNN) have been shown to provide a good so...
12/04/2021

PhishMatch: A Layered Approach for Effective Detection of Phishing URLs

Phishing attacks continue to be a significant threat on the Internet. Pr...
09/16/2022

SQ-Swin: a Pretrained Siamese Quadratic Swin Transformer for Lettuce Browning Prediction

Packaged fresh-cut lettuce is widely consumed as a major component of ve...

1 Introduction

Ransomware refers to a type of malicious program designed to block access to a computer system until a sum of money is paid, hence the name ransomware. The emergence and rapid increase of ransomware, compared to other types of malware, has been due to bitcoin and encryption technology. The invention of bitcoin has provided an anonymous payment channel for the criminals demanding ransoms. The wide use of strong encryption techniques in many applications also allowed the creation of malware where decryption is not possible without knowing the cryptographic key. The rapid increase in ransomware has been reported causing the loss of millions of dollars for businesses and individuals.

Statistic-based methods such as clustering zhu2021joint, entropy analysis mcintosh2019inadequacy; mcintosh2018large, similarity analysis Jang11, information flow analysis Mirzaei17, and examining manifest file rasthofer15 have been proposed to recognize malware quickly. However, the heavy reliance on manual analysis along with tools support became either too expensive or not possible due to the growing volume of ransomware attacks to millions within a very short period time.

The proliferation of machine learning techniques has allowed the reduction of manual intervention and has offered more automation-based machine analysis to rapidly recognize different types of ransomware (and other malware) to reduce a significantly increasing loss of money and productivity zhu2020multi; wei2021ae; xu2021improving; liu2021artificial

. Semi-automated approaches using random forest

Laurenza20; Laurenza17

were proposed to detect malware rapidly. For example, a KNN-based

Kirat13

and decision tree-based

amira21 classifiers to rapidly triage a large number of malware samples have been proposed in recent years.

However, there are still many hurdles in providing effective machine learning-based triage solutions. We use the term “classification” when we can clearly classify ransomware samples into specific known profiles of existing ransomware classes. Extending from the classification, we use the term “triage” to refer to the overall process of conducting an assessment not only classifying ransomware samples into known ransomware classes but also identifying new types of ransomware samples, for example, risky and unknown ransomware.

One big obstacle to the development of machine learning-based triage solutions is the availability of ransomware samples to train machine learning models to learn about the features involved in ransomware codes. Unfortunately, current machine learning techniques demand a very large dataset (e.g., hundreds and thousands) to train a machine learning technique, such that to recognize important/relevant features to detect different ransomware signatures or to find correlations across different features. The other obstacle is with feature representation when the quality of features feeding to machine learning models heavily decides the quality of outcomes (e.g., detection accuracy). In many cases, malware authors can still easily avoid detection by applying obfuscation techniques to change feature representations easily even after a machine learning model is trained with a set feature representation.

To address these issues, we propose a ransomware triage approach using a Siamese Neural Network (SNN). The main contributions of our proposed approach follow.

  • Our SNN based approach utilizes a pre-trained ResNet18 network in a meta-learning fashion to re-adjust the weights of features and the model parameters for constructing discriminative feature embedding space that is critical for classification. Using a pre-trained network allows our model to avoid creating weight and parameter biases typically associated with a model trained with a limited number of samples.

  • Our approach use entropy features directly obtained from the ransomware binary files. Compared to image features typically used in other malware triage applications, the use of entropy features retain better feature representation and are resilient to misclassification when ransomware samples contain noise from obfuscation.

  • The classification part of our approach can classify known profiled ransomware classes into their correct categories. Our experimental results show that our model can achieve classification accuracy of more than 88% and outperforms other similar classification-based triage approaches.

  • Using a weighted ratio based on similarity scores, our approach can triage the risky and unknown ransomware classes (i.e., possibly zero-day attack) that exhibit the features common for ransomware but do not exactly match known ransomware profiles. This risky and unknown ransomware can be prioritized for further analysis to formulate the right set of response strategies.

The rest of the paper is organized as follows. Section 2 presents related works. Section 3 provides the details of the proposed approach. Section 4 describes the details of the experiments and discusses the results, discussion, and limitations of the proposed approach. Finally, Section 5 draws a conclusion and describes the future work that is planned.

2 Related work

2.1 Triage Approach

Triage aims to assess the severity of the malware based on impact on the victim. The literature offers a variety of methods to speed up the severity of classification. The aim is to automatically investigate malware samples and pass them to the proper channels (e.g., cybersecurity professionals) for further analysis. A semi-automatic malware analysis architecture proposed in Laurenza20 replaces multi-class classification with a group of one-class classifiers to decrease the runtime. A random forest classifier is used in Laurenza17 with static malware features. Static features can be extracted fast without running the malware code and help perform quick triage. However, a weakness random forest algorithm as a classifier, this approach tends to overfit when there is malware noise (e.g., obfuscated code) intentionally introduced by malware authors alsouda2019iot.

BitShred Jang11 is a system developed for large-scale malware clustering and similarity analysis. BitShred uses feature hashing to reduce feature space dimensionality and uncovers family relations by investigating correlated features with Jaccard similarity. SigMal Kirat13 extracts features from the malware executable headers. SigMal transforms these features into a digital image. Signatures are extracted from images and investigated with similarity measures using the KNN algorithm. However, the KNN algorithm is susceptible to class imbalance was pointed out by kurniawati2018adaptive. TriDroid amira21 extracts coarse-grained features (e.g., permissions) to allocate an investigated android app into a three-class queue and uses fine-grained static features with three-class classifiers to confirm the queue assignments with high accuracy.

TRIFLOW Mirzaei17 uses information flow in Android apps to characterize behaviors of the risky apps for triage purposes. DROIDSEARCH rasthofer15 is a search engine with triage capability that triages based upton information such as ratings, downloads, or information from the manifest file. Mobile Application Security Triage (MAST) chakradeo2013mast uses statistical methods, called Multiple Correspondence Analysis (MCA), to find correlations on qualitative data obtained from the market. Rather than inspecting malware itself, tweet messages are investigated in Vinayakumar19 using Deep Neural Networks (DNN) for malware severity classification.

In the previous malware triage work, the proposed models have not achieved either satisfactory accuracy or were extensively tested on different attack classes. This leaves the triage system vulnerable only to classifying known malware samples with a high false-positive rate but also unable to identify unknown samples. The significance of the triage system is to support the automated operation of the system with less human intervention or cyber security analysts to recognize the urgency of the malicious attack to formulate more informed response strategies. Therefore, it is essential for a triage system to be equipped not only accurately classify them, but in some special cases, it identifies risky unknown malware for further analysis.

2.2 Transfer-Learning based meta Learning

The training strategy based on Transfer-learning differs from the episodic training strategy in the meta-learning. Instead, the conventional techniques are able to be applied in a pre-train model with the large amount of data from the base classes. And then, the pre-trained model is adapted to recognize the base classes and novel classes. yuan2020transfer; sun2019meta; bronskill2021memory; sun2020meta; soh2020meta. Yuan et al.yuan2020transfer proposed an offline adaptive learning algorithm that is able to be learned through the meta-training part and the fine-tuning part. The meta-training part aims to optimize the task procedure and fine-tuning part adjust the pre-trained parameters with the limited data in the new application task. Sun et al. sun2019meta learned a base-learner that could be adjusted to the new task with a few labeled samples. In order to improve the efficacy of performance, they further introduced the hard task meta-batch scheme as a learning curriculum. Soh et al. soh2020meta exploit both external and internal instances to present a Meta-Transfer Learning for zero-shot task.

Figure 1: Overview of The Triage system

3 Approach

3.1 Overview of Our System

The overview of our proposed model is shown in Fig. 1

. Our model consists of three phases, (1) feature preprocessing phase, (2) feature embedding phase, and (3) triage phase, respectively. The main goal of the feature preprocessing phase is to obtain entropy features from the bytecode of ransomware and preprocess them to feed to the model. In the feature embedding phase, our model constructs the feature embedding space of the input by utilizing two identical ResNet18 networks. We load a pre-trained version of the ResNet18 network trained on more than a million images from the ImageNet database

111ImageNet. http://www.image-net.org. This pre-trained ResNet18 is used in a meta-learning fashion where the weights and the model parameters are further adjusted for the entropy features. The triage phase provides a triage service that classifies different ransomware classes as well as identifies risky and unknown ransomware based on the weight ratio.

3.2 Feature Preprocessing Phase

Entropy is often used as the measure of changes in information content. More changes to the original information content produce higher entropy values while fewer changes to the original information are associated with lower entropy values. As discussed in zhu2022few; zhu2021task, using entropy values as feature representation has a number of advantages compared to using grayscale image features. The feature values associated with grayscale features can be easily fooled if malware authors apply certain obfuscation techniques. For instance, control flow obfuscation techniques wang2021efficient; azar2021data; chow2001approach can easily alter the control flow path of a malware program. For example, it inserts a junk code or shuffles the order of function calls which does not affect the semantics of the original malware program. This simple technique can effectively cause the appearance of grayscale images different from each other. This can easily lead to misclassification results where a classification algorithm puts them into different malware families when in fact they belong to the same malware family. In addition, the computation cost associated with grayscale image features is usually much higher due to the higher cost involved in processing texture features of grayscale images zhu2022few; zhu2021task. To avoid these weaknesses associated with grayscale features, we instead use entropy features that are more resilient to changes in the malware binary code and reduce computation costs.

To construct entropy features, we first read a ransomware binary file as a stream of bytes. The bytes are made into multiple segments each of which is comprised of 200 bytes. We further count the frequency of unique byte values, in the range of pixel values between 0 and 255, followed by computing the entropy using Shannon’s formula as seen in the following Eq. (1).

(1)

where

is the probability of an occurrence of a byte value. The entropy obtains the minimum value of 0 when all the byte values in a binary file are the same while the maximum entropy value of 8 is obtained when all the byte values are different. The entropy values are then concatenated as a stream of values and are used to generate an image-based entropy graph. The entropy graph, an image of size 224*224, is fed as entropy features as input to our model.

The process of generating the entropy features from a ransomware binary file is depicted in Algorithm 1.

Input : : malware binary file; : segment length; : the number of files
Output : entropy pattern image
1 while not reach to n do
2       1. read bytes from , and defined as a segment ; 2. for to do          2.1 compute the probability of appearing in using the Shannon entropy function 3. a entropy graph image generated from whole segments in a binary file 
Algorithm 1 Generating Entropy Features

3.3 Feature Concatenation Construction with Siamese Resnet18

We employ two identical ResNet18 networks as the backbone network to generate feature embeddings for each positive input (i.e., the entropy feature representing a ransomware byte code from the same family).

The ResNet18 network, a variation of the CNN-based architecture, we adopted has a single convolutional layer with a 7 7 kernel size and 4 blocks. All 4 blocks contain 4 identical convolution layers of the kernel size of 3

3 with ReLU activation functions. Each convolutional layer has the same dimension with a filter size of 64, 128, 256, and 512 in each block. A skip connection is made after the 2nd and 4th convolution layer of each block.

Each convolution layer is followed by a batch normalization layer which normalizes the input by subtracting the mean and dividing by the standard deviation to stabilize the input distribution of each layer of the network. In this way, the backbone network can accelerate the convergence and prevent gradient dispersion during the training process. Each batch normalization layer can also scale and shift the normalized data through learnable parameters to optimize the learning of the backbone network according to different input distributions.

Sequently, the entropy graph are provided as a image form of 2-dimensional vectors of a fixed size 224

224, the pre-trained ResNet18 is further trained and optimized to obtain updated weights and parameters. Adopting this style of meta-learning, our model can avoid the potential bias associated with a model trained with a limited number of training samples.

The four blocks of each ResNet18 are followed by two fully connected layers with a fattening layer in-between. The first fully connected layer has 512 neurons and the last fully connected layer serves as the output layer and has 256 neurons. The last fully connected layer from each Resnet18 is combined together to create a single fully connected layer with 512 neurons to compute a loss function across the features processed by two ResNet18 networks.

Feature embedding involves the transforming of features from the original space into a new space to support effective learning. To create more accurate feature embedding that represents the features of a ransomware family, we adopt two strategies. One is using an SNN where two identical ResNet18 networks are used to construct a feature embedding relative to each input where the common features from two inputs are combined. This strategy can improve the quality of features represented in a feature embedding as discussed in zhu2022few. Moreover, since different CNN architectures can capture diverse information in microscopic images, such concatenation of multiple CNN features integrates the information from different CNNs together to create a more discriminative feature representation compared with a single CNN structure. By combining two entropy features, the feature representation becomes not only richer but can address the discrete and sparse problem tang2013combining that arises from using a single feature. The other strategy we adopted is to train our model in a meta-learning fashion to utilize a pre-trained ResNet18 to adjust the weights and other model parameters according to entropy features.

3.4 Task Memory for Remaining Semantic Feature

Our aim is to create a semantic embedding invariance refer to the prototypical network snell2017prototypical and the can be constructed to a memory module called . We define the entries in as , in which each entries is an average feature performance computed via all instances in one class of the support set,

(2)

where is prototype for calculating relation to the novel set and is an instance of total number in the feature embedding of base set . Gao et al. gao2019hybrid

argued that a given instance may has a higher variance from the mean feature calculated by all of rest feature or some of them have been wrongly labeled. Moreover, the small number of sample tend to result in a large variance so that a straightly average performance may not estimate a truly distribution over the all instances.

(3)

where is a feature embedding generated by the linear layer, tanh denoted by is selected as an activation function to produce results among [-1,1] and means that the inner production of the two feature vectors of instances from the support set and query set respectively. And then, we aggregate the sequentially arriving semantic feature regardless of whose gradient and then average all the instance embedding in the support set for each relation,

(4)

where is the adaptive prototype for a class and indicates the class weights on each cluster. so that a high similar score given by the relation are obtained of higher weights to the instance in the support set.

(5)

where is the Euclidean distance function for a relation vectors according to the strategy of Snell et al. snell2017prototypical and it also generate a distribution over the specific class with the cross entropy loss function,

(6)

where classifier predicts which class the instance belong to according to the relation between in the query set and the in the memory . The vector of the dimension of 512 in the novel set (i.e., feature embedding) is compared with the features of feature embedding of a different class. Any bias in the class prediction is used as an error correction mechanism through the backward preparation.

3.5 Triage Phase

Our triage approach is similar to that of one used in the hospital system. Like patients whose symptoms clearly match the known profiles of illnesses, any (test) ransomware samples whose features clearly match the known ransomware classes are classified immediately to formulate a rapid response strategy. In some patient diagnosis whose symptoms does not clearly match one known profile but several profiles of illnesses, they could be sent to a designated specialist for further examinations. Our triage also recognizes this type of ransomware sample whose feature clearly demonstrates that it is ransomware but does not match a specific ransomware family. These cases are classified as risky and unknown ransomware so that further analysis is carried out rapidly to establish if this is a new variant (e.g., zero-day attack). Towards this approach, our model first uses similarity matching to search if a ransomware sample under the examination (i.e., test sample) exhibits features similar to any known ransomware classes. The level of similarity matching to different ransomware classes is computed as a weight ratio. By evaluating the weight ratio, we can triage a ransomware sample to if this can be classified as specific ransomware or should be classified as risky unknown ransomware.

3.5.1 Similarity Matching

In a general similarity searching approach, a query record is compared against a stored database of records. The main goal here is to retrieve a set of database records that are similar to the query record. For example, if there is a picture of a dog as a query record, a similarity search should give a list of pictures with dogs in them.

In the context of the triage approach, the database corresponds to a collection of vectors of ransomware, which is a collection of feature embedding trained for a pair of training samples from a ransomware class. Similarity searching in our context refers to searching for similar feature embeddings. We employ the FAISS algorithm 222https://github.com/facebookresearch/faiss/blob/main/INSTALL.md, Facebook’s library for faster similarity searching even for very large datasets, to calculate the similarity based on cosine distance. The details of the cosine distance we use are:

(7)

where is the number of samples, and are components of vector and respectively while is the product (dot) , and is the length, * is the cross product of the two vectors and .

In our model, we use the FAISS algorithm guo2003knn; ma2021pod to return the top most similar feature embeddings of ransomware samples whose cosine distance between the feature embedding of a test sample and the feature embeddings of the training sample are minima. Here, a similarity score indicates a different relationship between two samples (i.e., a test sample and a class representative from the trained ransomware classes). The minimum value of 0 indicates two samples are completely dissimilar. The maximum value of 1 indicates two samples are completely similar. The value close to 0 indicates the characteristics of orthogonality or decorrelation, while in-between values indicate intermediate similarity or dissimilarity.

Figure 2: How FAISS-based similarity matching works

Fig. 2 illustrates how FAISS-based similarity matching works. Let’s assume that the input is a pair of positive samples from the test set in which a feature embedding is constructed. The feature embedding is fed to the FAISS algorithm as a feature vector. The FAISS searches through the collection of feature embeddings from the training set which computes similarity scores based on cosine distance. The top feature embeddings with the highest similarity score are selected. Different weights are assigned based on the similarity score.

3.5.2 Weight Ratio and Classification

Based on the top search results and their weights, a weight ratio is computed and the final tirage/classification is done. Mathematically, weight to the th nearest neighbor for and classifies the test sample as the class that is assigned the most weight , as follows,

(8)

where denotes that the weights and is the total weights in the same class.

We normalize and regularize the weight ratio in a style that satisfies a linear interpolation with the maximum entropy (LiME) objective by linearly combining each weight

van2012kernel. This allows the combined weights to be better balanced and the test sample, , is best approximated based on the training samples through the Equation 9. The LiME objective is described as follows:

(9)

where is th training sample, the is the basis atoms for the feature vector, , and is the feature vector of test sample, , is a regularization parameter.

Fig. 3 illustrates how the weight ratios are computed to decide a triage/classification outcome. Let’s take a test sample whose top 5 search results were the 2 feature embeddings from the ransomware class3 and the 3 feature embeddings from the ransomware class1. The weights from the 2 feature embedding from the ransomware class3 were 5 (had the highest similarity score based on FAISS similarity matching) and 2 (had the 2nd lowest similarity score), respectively. This results in the total accumulated weight for the ransomware class3 = 7. Similarly, the weights from the 3 feature embedding from the ransomware class1 were 4, 3, and 1 which resulted in the total accumulated weight for the ransomware class1 = 8. The weight ratio is calculated by dividing the total accumulated weights for a class by the total weights of search results which results in a weight ratio of approximately 0.53 for class1 and 0.46 for class3. Let’s presume that the weight ratio rate we want to decide to classify a given sample is set at the threshold = 0.6. Neither the weight ratio of class1 nor class3 satisfies this threshold. In this case, our model will put the test sample in the “risky and unknown” class and the sample does not exhibit any clear features that can classify into a known ransomware class despite it exhibiting some common features from known ransomware classes. In contrast, take a test sample t2 where the weight ratio for class1 is equal to (or greater than) the threshold. In this case, the t2 is classified as belonging to the ransomware class1.

Figure 3: Weight Ratio and Triage

4 Experimental Results

This section describes the details of our experiments including the system environment, dataset, and performance metrics we used. We also discuss the experimental results with discussion.

4.1 Experimental Environment

This study was carried out using a 3.6 GHz 8-core Intel Core i7 processor with 32 GB memory on Windows 10 operating system. The proposed approach is developed using Python programming language with several statistical and visualization packages such as Sckit-learn, Numpy, Pandas, and Pytorch. Table 

1 summarizes the system configuration for our environment.

Unit Description
Processor 3.6 GHz 8-core Inter Core i7
RAM 32 GB
GPU GeForce GTX 1080 Ti
Operating System Windows 10
Packages Pytorch, Sckit-Learn, Numpy, Pandas
Table 1: System configuration

4.2 Dataset

We created a dataset containing ransomware binaries from ViruseShare 333VirusShare. https://virusshare.com/. The dataset comprises a total number of 1,048 samples from 11 families/classes of ransomware, each of which consists of a varying number of samples as listed in Table 2.

Training and Testing Instances Ratio (%)
Bitman 99 9.45
Cerber 91 8.68
Dalexis 9 0.86
Gandcrab 100 9.54
Locky 96 9.16
Petya 6 0.57
Teslacrypt 91 8.68
Upatre 18 1.72
Virlock 162 15.46
Risk Pool Test Instances Ratio (%)
Wannacry 178 16.98
Zerber 198 18.89
Table 2: Details of the ransomware dataset

The dataset is highly imbalanced as some classes, e.g., Petya and Dalexis, are largely outnumbered by the other classes, e.g., Virlock. We split the dataset into training and test datasets. The training contains 80% of the data from 9 classes while the test contains 20% of the data from all 11 classes. Two ransomware classes, Wannacry and Zerber, were never trained by our model but only used during the test phase to evaluate the triage capability of our model.

4.2.1 Performance Metrics

To evaluate the performance of our model, we used the balanced classification accuracy of F1 score, precision, and recall as performance metrics. Table

3

illustrates the confusion matrix.

Actual
Class Class
Predicted Class True Positive (TP) False Positive (FP)
Class False Negative (FN) True Negative (TN)
Table 3: Confusion matrix.

True Positive (TP) indicates a positive class correctly classified as a positive class. True Negative (TN) indicates a negative class correctly classified as a negative class. False Negative (FN) indicates a positive class is incorrectly classified as a negative class. False Positive (FP) indicates a negative class incorrectly classified as a positive class.

Based on the aforementioned terms, the evaluation metrics are calculated as follows:


(10)
(11)
(12)
(13)
[Learning rate] [Batch size]

[Hyperparameter

]
Figure 4: Hyperparameter Performance
(14)

4.3 Hyperparameter Performance

Figure 4 shows the performance of different hyperparameter tuning. As seen in the performance of learning rate (a), the learning rate = 0.0001 shows the best performance achieving the highest accuracy of over 80%. There is a significant difference in accuracy when other learning rates were used. The learning rate = 0.00001 achieved less than 20% accuracy while the learning rate = 0.001 improved the accuracy up to 60% but less than the learning rate = 0.0001. It shows that the learning rate was a significant hyperparameter influencing the overall accuracy compared to the center loss function. In terms of the performance of batch size, the best accuracy was achieved when the batch size = 64 was used. Similar accuracy rates all above 80% were achieved by the batch size = 32 and 128. We also tested the influence of hyperparameter that is used for computing center loss. Though provided a slightly better accuracy, all three rates contributed to achieving a high accuracy of around 90%. As these results illustrated the learning rate had the highest sensitivity to the model performance when compared to the batch size and hyperparameter .

In summary, Table 4 illustrates the best hyperparameters we used for our proposed model.

Hyperparameters Values Descriptions
Learning rate 0.0001 Learning speed (within range 0.0 and 1.0)
Batch size 64 No. of samples in one fwd/bwd pass
Epoch 50 No. of one fwd/bwd pass of all samples
 Center loss 0.1 Training loss rate
Table 4: Training parameters

4.4 Convergence Performance

Figure 5 illustrate the convergence loss and accuracy for both training and validation datasets. In terms of convergence loss shown in (a), the training of our model is done after 10 epochs and stabilizes until it reaches 50 epochs in both training and validation datasets confirming the training of our model is done at this stage. In terms of accuracy shown in (b), a high training and validation accuracy between 80% and 95% is achieved after 10 epochs where similar accuracy rates maintain until 50 epochs.

[Convergence loss] [Accuracy]

Figure 5: Convergence Performance

4.5 Classification Performance

We first check our model’s classification performance - that is, how well our model can classify ransomware samples that match the known profiles of existing ransomware. The results are shown in Table 5. For this benchmark, we used 9 ransomware classes depicted in Table 2 except Wannacry and Zerber. We applied a manual data argumentation technique to increase the sample size for three imbalance classes, Dalexis, Petya, and Uptatre, to have at least 30 samples each. We used 80% for training and 20% for testing. The existing methods were re-implemented using the entropy features based on our dataset to ensure the fairness of the evaluation. We ran the experiments on 10-fold validation - running ten times of different 80% and 20% spilt.

As it is shown, our approach exceeded existing approaches proposed for the classification-only based triage applications. The classification part of our triage application exceeds the performance of Decision Trees by 18.4%, SVM by 10.4%, KNN by 15.6%, and Random Forest by 10%. In all the metrics, our approach based on the classification outperformed other similar models while achieving accuracy of more than 88%.

Our model Decision SVM amira21 KNN Kirat13 Random
Trees amira21 Forest Laurenza20
Accuracy 88.6% 70.2% 78.2% 73.0% 78.6%
F1-score 87.2% 69.4% 77.8% 71.9% 78.1%
Precision 85.4% 69.9% 77.2% 73.8% 78.0%
Recall 88.6% 70.2% 78.2% 73.0% 78.6%
Table 5: Comparison to Similar Classification Models

Fig 6 illustrates the results based on AUC_ROC curve that shows the area under the curve for different ransomware classes in receiver operating characteristics (ROC).

The micro-average of all the classes is 0.98 and the macro-average of all the classes is 0.96. These high micro and macro averages indicate that our model has a good performance across the different classes. The highest value was observed for the “gradcrab” class with the perfect 100% (i.e., no false positive or false negative). Except for the “petya” class which recorded the lowest value at 0.85, all other classes reached above 91% values.

Figure 6: Performance based on AUC_ROC

4.6 Triage Performance

Figure 7 shows the projection of feature embedding space using the t-SNE dimensionality reduction technique. As seen in (a), the features from 9 different ransomware families contained in the training dataset are all intertwined together at Epoch = 1 (i.e., before training) which makes it difficult for our triage system to classify them into different classes. However, by Epoch = 50 seen in (b), there are distinct clusters formed around different ransomware where the unknown ransomware could not be placed into the known classes via the classifier, we could utilize the similarity compared with other features of samples from other families and put the unknown ransomware into the risky pool, as the (c) shows that the features of unknown ransomware are distinguished from other families. In other words, this is able to classify the unknown and known samples with high false-positive rates into the risky pool. Ransomware families with larger samples have bigger clusters while a small cluster is seen around the ransomware families with very small sample sizes (e.g., Dyslexia and Petya). This confirms that our proposed model is effective at training to distinguish different ransomware classes.

[Epoch 1]
[Epoch 50]

[Test embedding included of unknown ransomware]

Figure 7: T-SNE Visualization

We further examined the sensitivity of the threshold used for classification and accuracy, which is shown in Table 6. This is based on the similarity score obtained through running the KNN algorithm used in the FAISS similarity search with k size = 20. Here the threshold is the value associated with the normalized and regulized weight ratio.

Threshold Classified Risk Pool Accuracy
51.2% 48.8% 91.3 %
55.8% 44.2 % 89.1%
61.2% 38.8 % 78.4%
85.1% 14.9 % 65.5%
Table 6: Triage vs Accuracy based on Threshold

We take the weight ratio with higher confidence into account. Concretely speaking, when we used the highest threshold = 0.50, slightly more than half of the test samples = 51.2%, were fully classified into known ransomware classes while slightly less than half were classified as risky unknown ransomware. With such a high threshold rate, the accuracy rate was also very high because at this stage the similar score contributing to weight ratio calculation has to be very high (i.e., two samples being compared need to have a very high correlation). As expected, as the threshold value decreased, more test samples were classified into known ransomware classes as it require a lower level of similarity score computed between a test sample and one classified during training. The accuracy rate dropped orthogonally compared to the decrease of the threshold value. At the lowest threshold value = 0.30, 85.1 % of the test samples were classified into known malware classes while 14.9% of the test samples went into the risky and unknown ransomware class. The lowest accuracy rate of 65.5% was achieved at the lowest threshold rate.

4.7 Discussion and Limitations

In this study, we propose our model that can be used as a triage application based on the classification of ransomware. Instead of using image features, we utilize entropy features that are more robust against noises (e.g., changes made by obfuscation technique) and less computationally complex. We use pre-trained two ResNet18 networks in a meta-learning fashion to obtain accurate weights and other parameters when our training sample size is limited. Instead of a feature embedding created based on a single input, our model constructs a feature embedding based on two positive inputs from the twin ResNet18. This results in our feature embedding containing relevant features to detecting a certain ransomware class. Note that we only use positive inputs (i.e., two samples from the same ransomware family) to train our model to find more common features exhibited within the same ransomware family. This approach is different from other proposals where negative inputs (i.e., two samples from different ransomware families) are used chen2021exploring; wang2021fully; shao2021few. Our experimental results confirm that the use of positive inputs contributes to improving detection accuracy compare to the models using the mix of negative inputs. The triage part of our model utilizes weight ratio to compute a better classification result by taking into account the similarity scores produced by the feature vectors of test samples compared to the feature vectors of the whole training samples. This allows our triage system to classify known ransomware classes with high accuracy while being able to classify risky unknown ransomware. These risky unknown ransomware samples can be given a high priority for further analysis.

At the moment, our model simply classifies any samples that cannot be mapped into any known ransomware classes as the risky and unknown ransomware class as these samples exhibit the features of ransomware. However, it may require further deep analysis to investigate the true nature and severity of these samples (e.g., these may well be zero-day attack or maybe disguised through more complex obfuscation techniques based on the known ransomware classes).

We observed that there could be a potential bias when the feature vectors from unknown samples are made into feature embeddings trained with known samples. To reduce this bias, we may require further analysis of the nature of unknown samples before feeding them to our model.

Our model classifies any unseen test samples into the risky and unknown pool. However, it is possible that some of these samples may exhibit some features that appear in ransomware but it can be benign. Our model can be trained with some benign samples to understand the features associated with these types of samples to more clearly classify whether it is completely benign or risky ransomware.

By using entropy features, our model is more resilient to producing misclassification when obfuscated malware is included in training samples. Though the resilience against the control flow obfuscation technique has been evidenced in zhu2021task, the influence of other types of obfuscation techniques requires further investigation.

5 Conclusion

We proposed a ransomware triage approach that can rapidly classify different ransomware classes even in the presence of unknown classes. Our Siamese Neural Network (SNN) based approach utilizes a pre-trained ResNet18 in a meta-learning fashion to generate more accurate feature embedding and overcomes the biases in weight and parameter calculation typically associated with a model trained with training samples. Instead of image features typically used as inputs to machine learning-based malware detection and triage applications, our approach use entropy features directly obtained from the ransomware binary files. Our evaluations confirm that the use of entropy features provide a better feature representation and contribute towards improving triage accuracy. The experimental results tested on various ransomware samples show a very high classification accuracy exceeding 88%. In addition, we offer a new triage strategy that can recognize risky and unknown ransomware which exhibits the feature commonly seen in other known ransomware but exact matching profiles cannot be found. These types of ransomware can be easily prioritized for further analysis to formulate an appropriate response strategy faster before any significant damages emerge (e.g., the loss of ransom payments or reduced productivity).

We plan to extend our work to assign more sophisticated weights for the matching feature vectors for nearest neighbors as discussed in dorfer2018end; li2018end; yates2020capreolus and also include benign samples to compare their features with existing ransomware samples.

Acknowledgment

This research is supported by the Cyber Security Research Programme—Artificial Intelligence for Automating Response to Threats from the Ministry of Business, Innovation, and Employment (MBIE) of New Zealand as a part of the Catalyst Strategy Funds under the grant number MAUX1912.

References