B1902
Title: Inference for multiple data splitting and exchangeable p-values
Authors: Richard Guo - University of Cambridge (United Kingdom) [presenting]
Rajen D Shah - University of Cambridge (United Kingdom)
Abstract: Many modern procedures for hypothesis testing employ data splitting. Typically, the dataset is randomly split into two parts: certain complex nuisance functions are estimated from the first part, while the final statistic is computed by evaluating the estimated functions in the second part. Constructed as such, the errors from the two parts are independent, which is often essential to prevent ``double dipping'', controlling bias and ensuring asymptotic normality of the final statistic. However, such a practice has obvious drawbacks. First, the test is randomized and can yield inconsistent results on two analyses of the same data. Second, using only part of the sample hurts power. One remedy is to combine the statistics or p-values resulting from multiple data splits. We introduce a general method for large-sample inference of the combined statistic under minimal assumptions. We apply our method to a variety of problems: (1) testing conditional mean independence, (2) testing cluster structure in high dimensions and (3) testing no direct effect (Verma constraint) in a sequentially randomized trial. For these problems, our proposal is able to derandomize and improve power. Moreover, in contrast to existing p-value aggregation approaches that can be highly conservative, our method enjoys type-I error control that asymptotically approaches the nominal level.
