DeepAI AI Chat
Log In Sign Up

Methodological concerns about 'concordance-statistic for benefit' as a measure of discrimination in treatment benefit prediction

by   Yuan Xia, et al.
The University of British Columbia

Prediction algorithms that quantify the expected benefit of a given treatment conditional on patient characteristics can critically inform medical decisions. Quantifying the performance of such metrics is an active area of research. A recently proposed metric, the concordance statistic for benefit (cfb), evaluates the discriminatory ability of a treatment benefit predictor by directly extending the concept of the concordance statistic from a risk model for a binary outcome to a model for treatment benefit. In this work, we scrutinize cfb on multiple fronts. Through numerical examples and theoretical developments, we show that cfb is not a proper scoring rule. We also show that it is sensitive to the unestimable correlation between counterfactual outcomes, as well as to the matching algorithms for creating pairs. We argue that measures of statistical dispersion applied to predicted benefit do not suffer from these issues and can be an alternative metric for the discriminatory performance of treatment benefit predictors.


page 1

page 2

page 3

page 4


Novel metrics for quantifying the capacity of subgroup-defining variables to yield efficient treatment rules

A major objective of subgroup analysis in clinical trials is to explore ...

A Modified Net Reclassification Improvement Statistic

The continuous net reclassification improvement (NRI) statistic is a pop...

Function Driven Diffusion for Personalized Counterfactual Inference

We consider the problem of constructing diffusion operators high dimensi...