Interactive identification of individuals with positive treatment effect while controlling false discoveries

02/22/2021
by   Boyan Duan, et al.
0

Out of the participants in a randomized experiment with anticipated heterogeneous treatment effects, is it possible to identify which ones have a positive treatment effect, even though each has only taken either treatment or control but not both? While subgroup analysis has received attention, claims about individual participants are more challenging. We frame the problem in terms of multiple hypothesis testing: we think of each individual as a null hypothesis (the potential outcomes are equal, for example) and aim to identify individuals for whom the null is false (the treatment potential outcome stochastically dominates the control, for example). We develop a novel algorithm that identifies such a subset, with nonasymptotic control of the false discovery rate (FDR). Our algorithm allows for interaction – a human data scientist (or a computer program acting on the human's behalf) may adaptively guide the algorithm in a data-dependent manner to gain high identification power. We also propose several extensions: (a) relaxing the null to nonpositive effects, (b) moving from unpaired to paired samples, and (c) subgroup identification. We demonstrate via numerical experiments and theoretical analysis that the proposed method has valid FDR control in finite samples and reasonably high identification power.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/03/2023

Testing for idiosyncratic Treatment Effect Heterogeneity

This paper provides asymptotically valid tests for the null hypothesis o...
research
02/27/2019

Machine learning for subgroup discovery under treatment effect

In many practical tasks it is needed to estimate an effect of treatment ...
research
10/14/2021

Treatment Effect Detection with Controlled FDR under Dependence for Large-Scale Experiments

Online controlled experiments (also known as A/B Testing) have been view...
research
02/20/2020

Familywise Error Rate Control by Interactive Unmasking

We propose a method for multiple hypothesis testing with familywise erro...
research
09/02/2021

Online Testing of Subgroup Treatment Effects Based on Value Difference

Online A/B testing plays a critical role in the high-tech industry to gu...
research
06/03/2021

A New Measure of Overlap: An Alternative to the p–value

In this paper we present a new measure for the overlap of two density fu...
research
10/11/2022

A nonparametric relative treatment effect for direct comparisons of censored paired survival outcomes

A very classical problem in statistics is to test the stochastic superio...

Please sign up or login with your details

Forgot password? Click here to reset