You Can't Publish Replication Studies (and How to Anyways)

08/23/2019
by   Ghulam Jilani Quadri, et al.
0

Reproducibility has been increasingly encouraged by communities of science in order to validate experimental conclusions, and replication studies represent a significant opportunity to vision scientists wishing contribute new perceptual models, methods, or insights to the visualization community. Unfortunately, the notion of replication of previous studies does not lend itself to how we communicate research findings. Simple put, studies that re-conduct and confirm earlier results do not hold any novelty, a key element to the modern research publication system. Nevertheless, savvy researchers have discovered ways to produce replication studies by embedding them into other sufficiently novel studies. In this position paper, we define three methods – re-evaluation, expansion, and specialization – for embedding a replication study into a novel published work. Within this context, we provide a non-exhaustive case study on replications of Cleveland and McGill's seminal work on graphical perception. As it turns out, numerous replication studies have been carried out based on that work, which have both confirmed prior findings and shined new light on our understanding of human perception. Finally, we discuss how publishing a true replication study should be avoided, while providing suggestions for how vision scientists and others can still use replication studies as a vehicle to producing visualization research publications.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/27/2022

A Model-Based Approach for Specifying Changes in Replications of Empirical Studies in Computer Science

Context: The need of replicating empirical studies in Computer Science (...
research
11/02/2022

Can laypeople predict the replicability of social science studies without expert intervention: an exploratory study

The low replication rate of published studies has long concerned the soc...
research
08/08/2023

HotOS XIX Panel Report: Panel on Future of Reproduction and Replication of Systems Research

At HotOS XIX (2023), we organized a panel to discuss the future of repro...
research
03/22/2023

Overcoming Algorithm Aversion: A Comparison between Process and Outcome Control

Algorithm aversion occurs when humans are reluctant to use algorithms de...
research
11/26/2021

A Differentially Private Bayesian Approach to Replication Analysis

Replication analysis is widely used in many fields of study. Once a rese...
research
01/31/2021

Predicting replicability – analysis of survey and prediction market data from large-scale forecasting projects

The reproducibility of published research has become an important topic ...
research
08/03/2023

Replicability Study: Corpora For Understanding Simulink Models Projects

Background: Empirical studies on widely used model-based development too...

Please sign up or login with your details

Forgot password? Click here to reset