CMStatistics 2022: Start Registration
View Submission - CMStatistics
B0842
Title: Locally optimal design for A/B tests in the presence of covariates and network dependence Authors:  Lulu Kang - Illinois Institute of Technology (United States) [presenting]
Abstract: A/B test, a simple type of controlled experiment, refers to the statistical procedure of conducting an experiment to compare two treatments applied to test subjects. For example, many IT companies frequently conduct A/B tests on their users who are connected and form social networks. Often, the users' responses could be related to the network connection. We assume that the users, or the test subjects of the experiments, are connected on an undirected network, and the responses of two connected users are correlated. We include the treatment assignment, covariate features, and network connection in a conditional autoregressive model. Based on this model, we propose a design criterion that measures the variance of the estimated treatment effect and allocate the treatment settings to the test subjects by minimizing the criterion. Since the design criterion depends on an unknown network correlation parameter, we adopt the locally optimal design method and develop a hybrid optimization approach to obtain the optimal design. Through synthetic and real social network examples, we demonstrate the value of including network dependence in designing A/B experiments and validate that the proposed locally optimal design is robust to the choices of parameters.