Code Replicability in Computer Graphics

05/01/2020
by   Nicolas Bonneel, et al.
0

Being able to duplicate published research results is an important process of conducting research whether to build upon these findings or to compare with them. This process is called "replicability" when using the original authors' artifacts (e.g., code), or "reproducibility" otherwise (e.g., re-implementing algorithms). Reproducibility and replicability of research results have gained a lot of interest recently with assessment studies being led in various fields, and they are often seen as a trigger for better result diffusion and transparency. In this work, we assess replicability in Computer Graphics, by evaluating whether the code is available and whether it works properly. As a proxy for this field we compiled, ran and analyzed 151 codes out of 374 papers from 2014, 2016 and 2018 SIGGRAPH conferences. This analysis shows a clear increase in the number of papers with available and operational research codes with a dependency on the subfields, and indicates a correlation between code replicability and citation count. We further provide an interactive tool to explore our results and evaluation data.

READ FULL TEXT
research
09/30/2022

The state of play of reproducibility in Statistics: an empirical analysis

Reproducibility, the ability to reproduce the results of published paper...
research
04/16/2019

Go Wide, Go Deep: Quantifying the Impact of Scientific Papers through Influence Dispersion Trees

Despite a long history of use of citation count as a measure to assess t...
research
09/14/2019

A Step Toward Quantifying Independently Reproducible Machine Learning Research

What makes a paper independently reproducible? Debates on reproducibilit...
research
06/23/2022

Towards Better User Studies in Computer Graphics and Vision

Online crowdsourcing platforms make it easy to perform evaluations of al...
research
10/22/2020

Urban Sound Classification : striving towards a fair comparison

Urban sound classification has been achieving remarkable progress and is...
research
04/20/2023

A Study on Reproducibility and Replicability of Table Structure Recognition Methods

Concerns about reproducibility in artificial intelligence (AI) have emer...
research
07/28/2023

Lessons in Reproducibility: Insights from NLP Studies in Materials Science

Natural Language Processing (NLP), a cornerstone field within artificial...

Please sign up or login with your details

Forgot password? Click here to reset