Big data, big problems: Responding to "Are we there yet?"

09/02/2021
by   Alex Reinhart, et al.
0

Bradley et al. (arXiv:2106.05818v2), as part of an analysis of the performance of large-but-biased surveys during the COVID-19 pandemic, argue that the data defect correlation provides a useful tool to quantify the effects of sampling bias on survey results. We examine their analyses of results from the COVID-19 Trends and Impact Survey (CTIS) and show that, despite their claims, CTIS in fact performs well for its intended goals. Our examination reveals several limitations in the data defect correlation framework, including that it is only applicable for a single goal (population point estimation) and that it does not admit the possibility of measurement error. Through examples, we show that these limitations seriously affect the applicability of the framework for analyzing CTIS results. Through our own alternative analyses, we arrive at different conclusions, and we argue for a more expansive view of survey quality that accounts for the intended uses of the data and all sources of error, in line with the Total Survey Error framework that have been widely studied and implemented by survey methodologists.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/26/2023

Exploring the big data paradox for various estimands using vaccination data from the global COVID-19 Trends and Impact Survey (CTIS)

Selection bias poses a challenge to statistical inference validity in no...
research
06/10/2021

Are We There Yet? Big Data Significantly Overestimates COVID-19 Vaccination in the US

Public health efforts to control the COVID-19 pandemic rely on accurate ...
research
04/15/2021

A Critique of Differential Abundance Analysis, and Advocacy for an Alternative

It is largely taken for granted that differential abundance analysis is,...
research
10/13/2021

The effect of COVID-19 vaccinations on self-reported depression and anxiety during February 2021

Using the COVID-19 Trends and Impacts Survey (CTIS), we examine the effe...
research
02/28/2018

Epidemiologic analyses with error-prone exposures: Review of current practice and recommendations

Background: Variables in epidemiological observational studies are commo...
research
10/13/2022

We need to talk about nonprobability samples

It is well known that, in most circumstances, probability sampling is th...
research
07/31/2023

Using Proxy Pattern-Mixture Models to Explain Bias in Estimates of COVID-19 Vaccine Uptake from Two Large Surveys

Recently, attention was drawn to the failure of two very large internet-...

Please sign up or login with your details

Forgot password? Click here to reset