A Guide to Reproducible Research in Signal Processing and Machine Learning

by   Joseph Shenouda, et al.

Reproducibility is a growing problem that has been extensively studied among computational researchers and within the signal processing and machine learning research community. However, with the changing landscape of signal processing and machine learning research come new obstacles and unseen challenges in creating reproducible experiments. Due to these new challenges most experiments have become difficult, if not impossible, to be reproduced by an independent researcher. In 2016 a survey conducted by the journal Nature found that 50 researchers were unable to reproduce their own experiments. While the issue of reproducibility has been discussed in the literature and specifically within the signal processing community, it is still unclear to most researchers what are the best practices to ensure reproducibility without impinging on their primary responsibility of conducting research. We feel that although researchers understand the importance of making experiments reproducible, the lack of a clear set of standards and tools makes it difficult to incorporate good reproducibility practices in most labs. It is in this regard that we aim to present signal processing researchers with a set of practical tools and strategies that can help mitigate many of the obstacles to producing reproducible computational experiments.


page 6

page 11

page 13

page 14


Reproducibility in Machine Learning-Driven Research

Research is facing a reproducibility crisis, in which the results and fi...

Perspectives on Machine Learning from Psychology's Reproducibility Crisis

In the early 2010s, a crisis of reproducibility rocked the field of psyc...

Collaborative Intelligence: Challenges and Opportunities

This paper presents an overview of the emerging area of collaborative in...

Interactive Tools for Reproducible Science – Understanding, Supporting, and Motivating Reproducible Science Practices

Reproducibility should be a cornerstone of science as it enables validat...

Graph signal processing for machine learning: A review and new perspectives

The effective representation, processing, analysis, and visualization of...

SIERRA: A Modular Framework for Research Automation and Reproducibility

Modern intelligent systems researchers form hypotheses about system beha...

The Dagstuhl Beginners Guide to Reproducibility for Experimental Networking Research

Reproducibility is one of the key characteristics of good science, but h...

Please sign up or login with your details

Forgot password? Click here to reset