The Least Difference in Means: A Statistic for Effect Size Strength and Practical Significance

05/24/2022
by   Bruce A. Corliss, et al.
0

With limited resources, scientific inquiries must be prioritized for further study, funding, and translation based on their practical significance: whether the effect size is large enough to be meaningful in the real world. Doing so must evaluate a result's effect strength, defined as a conservative assessment of practical significance. We propose the least difference in means (δ_L) as a two-sample statistic that can quantify effect strength and perform a hypothesis test to determine if a result has a meaningful effect size. To facilitate consensus, δ_L allows scientists to compare effect strength between related results and choose different thresholds for hypothesis testing without recalculation. Both δ_L and the relative δ_L outperform other candidate statistics in identifying results with higher effect strength. We use real data to demonstrate how the relative δ_L compares effect strength across broadly related experiments. The relative δ_L can prioritize research based on the strength of their results.

READ FULL TEXT

Authors

page 27

page 29

01/04/2022

The Most Difference in Means: A Statistic for Null and Near-Zero Results

Two-sample p-values test for statistical significance. Yet p-values cann...
01/10/2022

Knots and their effect on the tensile strength of lumber

When assessing the strength of sawn lumber for use in engineering applic...
01/28/2019

Effect sizes of the differences between means without assuming the variance equality and between a mean and a constant

Hedges' d, an existing unbiased effect size of the difference between me...
03/01/2019

Improved Differentially Private Analysis of Variance

Hypothesis testing is one of the most common types of data analysis and ...
06/22/2018

Hypothesis testing near singularities and boundaries

The likelihood ratio statistic, with its asymptotic χ^2 distribution at ...
11/02/2020

p-value peeking and estimating extrema

A pervasive issue in statistical hypothesis testing is that the reported...
01/14/2021

Evaluating prediction systems in software project estimation

Context: Software engineering has a problem in that when we empirically ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.