Do I really need all this work to find vulnerabilities? An empirical case study comparing vulnerability detection techniques on a Java application

by   Sarah Elder, et al.
NC State University

CONTEXT: Applying vulnerability detection techniques is one of many tasks using the limited resources of a software project. OBJECTIVE: The goal of this research is to assist managers and other decision-makers in making informed choices about the use of software vulnerability detection techniques through an empirical study of the efficiency and effectiveness of four techniques on a Java-based web application. METHOD: We apply four different categories of vulnerability detection techniques –  systematic manual penetration testing (SMPT), exploratory manual penetration testing (EMPT), dynamic application security testing (DAST), and static application security testing (SAST) – to an open-source medical records system. RESULTS: We found the most vulnerabilities using SAST. However, EMPT found more severe vulnerabilities. With each technique, we found unique vulnerabilities not found using the other techniques. The efficiency of manual techniques (EMPT, SMPT) was comparable to or better than the efficiency of automated techniques (DAST, SAST) in terms of Vulnerabilities per Hour (VpH). CONCLUSIONS: The vulnerability detection technique practitioners should select may vary based on the goals and available resources of the project. If the goal of an organization is to find "all" vulnerabilities in a project, they need to use as many techniques as their resources allow.


Vulnerability Detection is Just the Beginning

Vulnerability detection plays a key role in secure software development....

Vulnerability Prioritization: An Offensive Security Approach

Organizations struggle to handle sheer number of vulnerabilities in thei...

Predicting Exploitation of Disclosed Software Vulnerabilities Using Open-source Data

Each year, thousands of software vulnerabilities are discovered and repo...

An Empirical Study on Benchmarks of Artificial Software Vulnerabilities

Recently, various techniques (e.g., fuzzing) have been developed for vul...

Understanding the Quality of Container Security Vulnerability Detection Tools

Virtualization enables information and communications technology industr...

Structuring a Comprehensive Software Security Course Around the OWASP Application Security Verification Standard

Lack of security expertise among software practitioners is a problem wit...

Please sign up or login with your details

Forgot password? Click here to reset