ORIE Special Seminar: Hannah Li (Stanford)

Description

Experimental Design in Two-Sided Marketplaces: Competition and Interference

Two-sided marketplace platforms often run experiments to test the effect of an intervention before launching it platform-wide. However, estimates of the treatment effect obtained in these experiments can be biased, due to interference arising from marketplace competition. These biases can negatively impact a platform’s ability to make data-driven decisions. We develop market models to capture market dynamics and use the model to investigate the effect of inference on the performance of different designs and estimators. First, we show that the bias of commonly used estimators (in demand-side and supply-side randomized designs) depends on market balance. Second, we introduce a novel class of experimental designs based on two-sided randomization (TSR) and propose estimators with lower bias across wide ranges of market balance. Finally, we evaluate the impact of these biases in practice, through calibrated simulations to platform data and an implementation of the TSR design in a marketplace platform.

Bio:
Hannah Li is a Ph.D. candidate at Stanford University, where she is advised by Ramesh Johari and Gabriel Weintraub. She is part of the Operations Research group in MS&E and the Society and Algorithms Lab. Her research uses math modeling, optimization, and causal inference in order to analyze and design data science methodology for marketplace platforms. She has worked with several platforms, including Airbnb, Common App, Opendoor and Vinted. Before coming to Stanford, she graduated from Pomona College with a degree in mathematics.