The Least Difference in Means: A Statistic for Effect Size Strength and Practical Significance

by   Bruce A. Corliss, et al.

With limited resources, scientific inquiries must be prioritized for further study, funding, and translation based on their practical significance: whether the effect size is large enough to be meaningful in the real world. Doing so must evaluate a result's effect strength, defined as a conservative assessment of practical significance. We propose the least difference in means (δ_L) as a two-sample statistic that can quantify effect strength and perform a hypothesis test to determine if a result has a meaningful effect size. To facilitate consensus, δ_L allows scientists to compare effect strength between related results and choose different thresholds for hypothesis testing without recalculation. Both δ_L and the relative δ_L outperform other candidate statistics in identifying results with higher effect strength. We use real data to demonstrate how the relative δ_L compares effect strength across broadly related experiments. The relative δ_L can prioritize research based on the strength of their results.



page 27

page 29


The Most Difference in Means: A Statistic for Null and Near-Zero Results

Two-sample p-values test for statistical significance. Yet p-values cann...

Knots and their effect on the tensile strength of lumber

When assessing the strength of sawn lumber for use in engineering applic...

Effect sizes of the differences between means without assuming the variance equality and between a mean and a constant

Hedges' d, an existing unbiased effect size of the difference between me...

Improved Differentially Private Analysis of Variance

Hypothesis testing is one of the most common types of data analysis and ...

Hypothesis testing near singularities and boundaries

The likelihood ratio statistic, with its asymptotic χ^2 distribution at ...

p-value peeking and estimating extrema

A pervasive issue in statistical hypothesis testing is that the reported...

Evaluating prediction systems in software project estimation

Context: Software engineering has a problem in that when we empirically ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.