Can We Trust Tests To Automate Dependency Updates? A Case Study of Java Projects

by   Joseph Hejderup, et al.

Developers are increasingly using services such as Dependabot to automate dependency updates. However, recent research has shown that developers perceive such services as unreliable, as they heavily rely on test coverage to detect conflicts in updates. To understand the prevalence of tests exercising dependencies, we calculate the test coverage of direct and indirect uses of dependencies in 521 well-tested Java projects. We find that tests only cover 58 artificial updates with simple faults covering all dependency usages in 262 projects, we measure the effectiveness of test suites in detecting semantic faults in dependencies; we find that tests can only detect 47 35 investigate the use of change impact analysis as a means of reducing false negatives; on average, our tool can uncover 74 dependencies and 64 test suites. We then apply our tool in 22 real-world dependency updates, where it identifies three semantically conflicting cases and five cases of unused dependencies. Our findings indicate that the combination of static and dynamic analysis should be a requirement for future dependency updating systems.


page 1

page 2

page 3

page 4


A Longitudinal Analysis of Bloated Java Dependencies

We study the evolution and impact of bloated dependencies in a single so...

Automating Dependency Updates in Practice: An Exploratory Study on GitHub Dependabot

Dependency management bots automatically open pull requests to update so...

E2E Web Test Dependency Detection using NLP

E2E web test suites are prone to test dependencies due to the heterogene...

Towards Integration-Level Test Case Generation Using Call Site Information

Search-based approaches have been used in the literature to automate the...

Analyzing Impact of Dependency Injection on Software Maintainability

Dependency injection (DI) is generally known to improve maintainability ...

The Used, the Bloated, and the Vulnerable: Reducing the Attack Surface of an Industrial Application

Software reuse may result in software bloat when significant portions of...

Please sign up or login with your details

Forgot password? Click here to reset