Using Machine Learning to Test Causal Hypotheses in Conjoint Analysis

by   Dae Woong Ham, et al.

Conjoint analysis is a popular experimental design used to measure multidimensional preferences. Researchers examine how varying a factor of interest, while controlling for other relevant factors, influences decision-making. Currently, there exist two methodological approaches to analyzing data from a conjoint experiment. The first focuses on estimating the average marginal effects of each factor while averaging over the other factors. Although this allows for straightforward design-based estimation, the results critically depend on the distribution of other factors and how interaction effects are aggregated. An alternative model-based approach can compute various quantities of interest, but requires researchers to correctly specify the model, a challenging task for conjoint analysis with many factors and possible interactions. In addition, a commonly used logistic regression has poor statistical properties even with a moderate number of factors when incorporating interactions. We propose a new hypothesis testing approach based on the conditional randomization test to answer the most fundamental question of conjoint analysis: Does a factor of interest matter in any way given the other factors? Our methodology is solely based on the randomization of factors, and hence is free from assumptions. Yet, it allows researchers to use any test statistic, including those based on complex machine learning algorithms. As a result, we are able to combine the strengths of the existing design-based and model-based approaches. We illustrate the proposed methodology through conjoint analysis of immigration preferences and political candidate evaluation. We also extend the proposed approach to test for regularity assumptions commonly used in conjoint analysis.


page 1

page 2

page 3

page 4


DECoVaC: Design of Experiments with Controlled Variability Components

Reproducible research in Machine Learning has seen a salutary abundance ...

Confirmatory Factor Analysis -- A Case study

Confirmatory Factor Analysis (CFA) is a particular form of factor analys...

Workflow Techniques for the Robust Use of Bayes Factors

Inferences about hypotheses are ubiquitous in the cognitive sciences. Ba...

A Conditional Randomization Test for Sparse Logistic Regression in High-Dimension

Identifying the relevant variables for a classification model with corre...

Design and analysis of a microplate assay in the presence of multiple restrictions on the randomization

Experiments using multi-step protocols often involve several restriction...

Experimental Evaluation of Individualized Treatment Rules

In recent years, the increasing availability of individual-level data an...

The Bradly-Terry Regression Trunk approach for modelling preference data with small trees

This paper introduces the Bradley-Terry Regression Trunk model, a novel ...

Please sign up or login with your details

Forgot password? Click here to reset