Redefining Populations of Inference for Generalizations from Small Studies

04/29/2022
by   Wendy Chan, et al.
0

With the growth in experimental studies in education, policymakers and practitioners are interested in understanding not only what works, but for whom an intervention works. This interest in the generalizability of a study's findings has benefited from advances in statistical methods that aim to improve generalizations, particularly when the original study sample is not randomly selected. A challenge, however, is that generalizations are frequently based on small study samples. Limited data affects both the precision and bias of treatment impact estimates, calling into question the validity of generalizations. This study explores the extent to which redefining the inference population is a useful tool to improve generalizations from small studies. We discuss two main frameworks for redefining populations and apply the methods to an empirical example based on a completed cluster randomized trial in education. We discuss the implications of various methods to redefine the population and conclude with guidance and some recommendations for practitioners interested in using redefinition.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/09/2019

Making Study Populations Visible through Knowledge Graphs

Treatment recommendations within Clinical Practice Guidelines (CPGs) are...
research
03/17/2020

Generalizing Randomized Trial Findings to a Target Population using Complex Survey Population Data

Randomized trials are considered the gold standard for estimating causal...
research
06/28/2021

What to do if N is two?

The field of in-vivo neurophysiology currently uses statistical standard...
research
03/05/2015

External Validity: From Do-Calculus to Transportability Across Populations

The generalizability of empirical findings to new environments, settings...
research
07/31/2020

Causal inference methods for small non-randomized studies: Methods and an application in COVID-19

The usual development cycles are too slow for the development of vaccine...
research
01/23/2021

The Role of Distributional Overlap on the Precision Gain of Bounds for Generalization

Over the past ten years, propensity score methods have made an important...

Please sign up or login with your details

Forgot password? Click here to reset