DeepAI AI Chat
Log In Sign Up

Reproducibility in Machine Learning for Health

by   Matthew B. A. McDermott, et al.
Evidation Health
NYU college

Machine learning algorithms designed to characterize, monitor, and intervene on human health (ML4H) are expected to perform safely and reliably when operating at scale, potentially outside strict human supervision. This requirement warrants a stricter attention to issues of reproducibility than other fields of machine learning. In this work, we conduct a systematic evaluation of over 100 recently published ML4H research papers along several dimensions related to reproducibility. We find that the field of ML4H compares poorly to more established machine learning fields, particularly concerning data and code accessibility. Finally, drawing from success in other fields of science, we propose recommendations to data providers, academic publishers, and the ML4H research community in order to promote reproducible research moving forward.


Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)

One of the challenges in machine learning research is to ensure that pre...

Perspectives on Machine Learning from Psychology's Reproducibility Crisis

In the early 2010s, a crisis of reproducibility rocked the field of psyc...

Reproducibility in machine learning for medical imaging

Reproducibility is a cornerstone of science, as the replication of findi...

Reproducibility of Machine Learning: Terminology, Recommendations and Open Issues

Reproducibility is one of the core dimensions that concur to deliver Tru...

A Step Toward Quantifying Independently Reproducible Machine Learning Research

What makes a paper independently reproducible? Debates on reproducibilit...

Reproducibility in medical image radiomic studies: contribution of dynamic histogram binning

The de facto standard of dynamic histogram binning for radiomic feature ...

Recommendations for Systematic Research on Emergent Language

Emergent language is unique among fields within the discipline of machin...