EcoSta 2018: Registration
View Submission - EcoSta2018
Title: Adaptive filtering procedures for replicability analysis of high-throughput experiments Authors:  Jingshu Wang - University of Pennsylvania (United States) [presenting]
Weijie Su - The Wharton School, University of Pennsylvania (United States)
Chiara Sabatti - Stanford University (United States)
Art Owen - Stanford University (United States)
Abstract: Replicability is a fundamental quality of scientific discoveries. While meta-analysis provides a framework to evaluate the strength of signals across multiple studies accounting for experimental variability, it does not investigate replicability. A single, possibly non-reproducible study, can be enough to bring significance. In contrast, the partial conjunction (PC) alternative hypothesis stipulates that for a chosen number $r$ ($r > 1$), at least $r$ out of $n$ related individual hypotheses are non-null, making it a useful measure of replicability. Motivated by genetics problems, we consider settings where a large number $M$ of partial conjunction null hypotheses are tested, using an $n\times M$ matrix of $p$-values where $n$ is the number of studies. Applying multiple testing adjustments directly to PC $p$-values can be very conservative. We here introduce AdaFilter, a new procedure that, mindful of the fact that the PC null is a composite hypothesis, increases power by filtering out unlikely candidate PC hypotheses using the whole $p$-value matrix. We prove that appropriate versions of AdaFilter control the familywise error rate and the per family error rate under independence. We show that these error rates and the false discovery rate can be controlled under independence and a within-study local dependence structure while achieving much higher power than existing methods. We illustrate the effectiveness of the AdaFilter procedures with three different case studies.