The Identification and Estimation of Direct and Indirect Effects in A/B Tests through Causal Mediation Analysis

06/24/2019
by   Xuan Yin, et al.
0

E-commerce companies have a number of online products, such as organic search, sponsored search, and recommendation modules, to fulfill customer needs. Although each of these products provides a unique opportunity for users to interact with a portion of the overall inventory, they are all similar channels for users and compete for limited time and monetary budgets of users. To optimize users' overall experiences on an E-commerce platform, instead of understanding and improving different products separately, it is important to gain insights into the evidence that a change in one product would induce users to change their behaviors in others, which may be due to the fact that these products are functionally similar. In this paper, we introduce causal mediation analysis as a formal statistical tool to reveal the underlying causal mechanisms. Existing literature provides little guidance on cases where multiple unmeasured causally-dependent mediators exist, which are common in A/B tests. We seek a novel approach to identify in those scenarios direct and indirect effects of the treatment. In the end, we demonstrate the effectiveness of the proposed method in data from Etsy's real A/B tests and shed lights on complex relationships between different products.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/29/2018

Mediation Analysis in Online Experiments at Booking.com: Disentangling Direct and Indirect Effects

Online experimentation is at the core of Booking.com's customer-centric ...
research
03/11/2019

Estimating Individual Advertising Effect in E-Commerce

Online advertising has been the major monetization approach for Internet...
research
06/26/2018

A NoSQL Data-based Personalized Recommendation System for C2C e-Commerce

With the considerable development of customer-to-customer (C2C) e-commer...
research
04/12/2021

Interpretable Methods for Identifying Product Variants

For e-commerce companies with large product selections, the organization...
research
12/14/2021

Measuring Equity: Funnel Representation Measurement

We present a methodology to measure the gender representation for online...
research
04/26/2023

Categorising Products in an Online Marketplace: An Ensemble Approach

In recent years, product categorisation has been a common issue for E-co...
research
07/10/2022

Not Just Skipping. Understanding the Effect of Sponsored Content on Users' Decision-Making in Online Health Search

Advertisements (ads) are an innate part of search engine business models...

Please sign up or login with your details

Forgot password? Click here to reset