DeepAI AI Chat
Log In Sign Up

Trimmed Match Design for Randomized Paired Geo Experiments

by   Aiyou Chen, et al.

How to measure the incremental Return On Ad Spend (iROAS) is a fundamental problem for the online advertising industry. A standard modern tool is to run randomized geo experiments, where experimental units are non-overlapping ad-targetable geographical areas (Vaver Koehler 2011). However, how to design a reliable and cost-effective geo experiment can be complicated, for example: 1) the number of geos is often small, 2) the response metric (e.g. revenue) across geos can be very heavy-tailed due to geo heterogeneity, and furthermore 3) the response metric can vary dramatically over time. To address these issues, we propose a robust nonparametric method for the design, called Trimmed Match Design (TMD), which extends the idea of Trimmed Match (Chen Au 2019) and furthermore integrates the techniques of optimal subset pairing and sample splitting in a novel and systematic manner. Some simulation and real case studies are presented. We also point out a few open problems for future research.


page 1

page 2

page 3

page 4


Robust Causal Inference for Incremental Return on Ad Spend with Randomized Geo Experiments

Evaluating the incremental return on ad spend (iROAS) of a prospective o...

Bias Correction For Paid Search In Media Mix Modeling

Evaluating the return on ad spend (ROAS), the causal effect of advertisi...

On Designing a Two-stage Auction for Online Advertising

For the scalability of industrial online advertising systems, a two-stag...

MM2RTB: Bringing Multimedia Metrics to Real-Time Bidding

In display advertising, users' online ad experiences are important for t...

Mitigating Divergence of Latent Factors via Dual Ascent for Low Latency Event Prediction Models

Real-world content recommendation marketplaces exhibit certain behaviors...

Stratifying Online Field Experiments Using the Pigeonhole Design

Practitioners and academics have long appreciated the benefits that expe...

Measuring Average Treatment Effect from Heavy-tailed Data

Heavy-tailed metrics are common and often critical to product evaluation...