Custodes: Auditable Hypothesis Testing

01/19/2019
by   Sacha Servan-Schreiber, et al.
0

We present Custodes: a new approach to solving the complex issue of preventing "p-hacking" in scientific studies. The novel protocol provides a concrete and publicly auditable method for controlling false-discoveries and eliminates any potential for data dredging on the part of researchers during data-analysis phase. Custodes provides provable guarantees on the validity of each hypotheses test performed on a dataset by using cryptographic techniques to certify outcomes of statistical tests. Custodes achieves this using a decentralized authority and a tamper-proof ledger which enables the auditing of the hypothesis testing process. We present a construction of Custodes which we implement and evaluate using both real and synthetic datasets on common statistical tests, demonstrating the effectiveness and practicality of Custodes in the real world.

READ FULL TEXT
research
02/27/2020

PAPRIKA: Private Online False Discovery Rate Control

In hypothesis testing, a false discovery occurs when a hypothesis is inc...
research
10/11/2018

A Simple Way to Deal with Cherry-picking

Statistical hypothesis testing serves as statistical evidence for scient...
research
03/06/2016

Classical Statistics and Statistical Learning in Imaging Neuroscience

Neuroimaging research has predominantly drawn conclusions based on class...
research
01/24/2021

NeurT-FDR: Controlling FDR by Incorporating Feature Hierarchy

Controlling false discovery rate (FDR) while leveraging the side informa...
research
08/10/2021

Why multiple hypothesis test corrections provide poor control of false positives in the real world

Most scientific disciplines use significance testing to draw conclusions...
research
09/17/2023

ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing

Evaluating outputs of large language models (LLMs) is challenging, requi...
research
01/05/2018

On the Logic (plus some history and philosophy) of Statistical Tests and Scientific Investigation

Every scientific endeavour consists of (at least) two components: A hypo...

Please sign up or login with your details

Forgot password? Click here to reset