Don't Pick the Cherry: An Evaluation Methodology for Android Malware Detection Methods

03/25/2019
by   Aleieldin Salem, et al.
0

In evaluating detection methods, the malware research community relies on scan results obtained from online platforms such as VirusTotal. Nevertheless, given the lack of standards on how to interpret the obtained data to label apps, researchers hinge on their intuitions and adopt different labeling schemes. The dynamicity of VirusTotal's results along with adoption of different labeling schemes significantly affect the accuracies achieved by any given detection method even on the same dataset, which gives subjective views on the method's performance and hinders the comparison of different malware detection techniques. In this paper, we demonstrate the effect of varying (1) time, (2) labeling schemes, and (3) attack scenarios on the performance of an ensemble of Android repackaged malware detection methods, called dejavu, using over 30,000 real-world Android apps. Our results vividly show the impact of varying the aforementioned 3 dimensions on dejavu's performance. With such results, we encourage the adoption of a standard methodology that takes into account those 3 dimensions in evaluating newly-devised methods to detect Android (repackaged) malware.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/01/2020

Towards Accurate Labeling of Android Apps for Reliable Malware Detection

In training their newly-developed malware detection methods, researchers...
research
05/12/2020

Ransomware in Windows and Android Platforms

Malware proliferation and sophistication have drastically increased and ...
research
05/31/2022

Dataset Bias in Android Malware Detection

Researchers have proposed kinds of malware detection methods to solve th...
research
07/01/2020

Maat: Automatically Analyzing VirusTotal for Accurate Labeling and Effective Malware Detection

The malware analysis and detection research community relies on the onli...
research
05/29/2021

BPFroid: Robust Real Time Android Malware Detection Framework

We present BPFroid – a novel dynamic analysis framework for Android that...
research
05/25/2022

Towards a Fair Comparison and Realistic Design and Evaluation Framework of Android Malware Detectors

As in other cybersecurity areas, machine learning (ML) techniques have e...
research
08/13/2019

Similarity-based Android Malware Detection Using Hamming Distance of Static Binary Features

In this paper, we develop four malware detection methods using Hamming d...

Please sign up or login with your details

Forgot password? Click here to reset