The Role of Publicly Available Data in MICCAI Papers from 2014 to 2018

by   Nicholas Heller, et al.

Widely-used public benchmarks are of huge importance to computer vision and machine learning research, especially with the computational resources required to reproduce state of the art results quickly becoming untenable. In medical image computing, the wide variety of image modalities and problem formulations yields a huge task-space for benchmarks to cover, and thus the widespread adoption of standard benchmarks has been slow, and barriers to releasing medical data exacerbate this issue. In this paper, we examine the role that publicly available data has played in MICCAI papers from the past five years. We find that more than half of these papers are based on private data alone, although this proportion seems to be decreasing over time. Additionally, we observed that after controlling for open access publication and the release of code, papers based on public data were cited over 60 private-data counterparts. Further, we found that more than 20 public data did not provide a citation to the dataset or associated manuscript, highlighting the "second-rate" status that data contributions often take compared to theoretical ones. We conclude by making recommendations for MICCAI policies which could help to better incentivise data sharing and move the field toward more efficient and reproducible science.


page 4

page 5


What is the State of the Art of Computer Vision-Assisted Cytology? A Systematic Literature Review

Cytology is a low-cost and non-invasive diagnostic procedure employed to...

The state of play of reproducibility in Statistics: an empirical analysis

Reproducibility, the ability to reproduce the results of published paper...

S2ORC: The Semantic Scholar Open Research Corpus

We introduce S2ORC, a large contextual citation graph of English-languag...

Ranking State-of-the-art Papers via Incomplete Tournaments Induced by Citations from Performance Tables

How can we find state-of-the-art papers for a given task? Is it possible...

A dataset for Computer-Aided Detection of Pulmonary Embolism in CTA images

Todays, researchers in the field of Pulmonary Embolism (PE) analysis nee...

Generating Local Maps of Science using Deep Bibliographic Coupling

Bibliographic and co-citation coupling are two analytical methods widely...

A Gold Standard Dataset for the Reviewer Assignment Problem

Many peer-review venues are either using or looking to use algorithms to...

Please sign up or login with your details

Forgot password? Click here to reset