DeepAI AI Chat
Log In Sign Up

TRANSMUT-SPARK: Transformation Mutation for Apache Spark

by   Joao Batista de Souza Neto, et al.

We propose TRANSMUT-Spark, a tool that automates the mutation testing process of Big Data processing code within Spark programs. Apache Spark is an engine for Big Data Processing. It hides the complexity inherent to Big Data parallel and distributed programming and processing through built-in functions, underlying parallel processes, and data management strategies. Nonetheless, programmers must cleverly combine these functions within programs and guide the engine to use the right data management strategies to exploit the large number of computational resources required by Big Data processing and avoid substantial production losses. Many programming details in data processing code within Spark programs are prone to false statements that need to be correctly and automatically tested. This paper explores the application of mutation testing in Spark programs, a fault-based testing technique that relies on fault simulation to evaluate and design test sets. The paper introduces the TRANSMUT-Spark solution for testing Spark programs. TRANSMUT-Spark automates the most laborious steps of the process and fully executes the mutation testing process. The paper describes how the tool automates the mutants generation, test execution, and adequacy analysis phases of mutation testing with TRANSMUT-Spark. It also discusses the results of experiments that were carried out to validate the tool to argue its scope and limitations.


page 20

page 38

page 39


An Abstract View of Big Data Processing Programs

This paper proposes a model for specifying data flow based parallel data...

Mull it over: mutation testing based on LLVM

This paper describes Mull, an open-source tool for mutation testing base...

Securing Big Data systems, A cybersecurity management discussion

This paper explores the essential areas of cybersecurity management for ...

Big Data Refinement

"Big data" has become a major area of research and associated funding, a...

Testing Quantum Programs with Multiple Subroutines

Errors in quantum programs are challenging to track down due to the unce...

Parallel-and-stream accelerator for computationally fast supervised learning

Two dominant distributed computing strategies have emerged to overcome t...

Measuring Coverage of Prolog Programs Using Mutation Testing

Testing is an important aspect in professional software development, bot...