A1217
Title: A two-stage subsampling approach for big data
Authors: Vasilis Chasiotis - Athens University of Economics and Business (Greece) [presenting]
Lin Wang - Purdue University (United States)
Dimitris Karlis - RC Athens University of Economics and Business (Greece)
Abstract: In big data analytics, there is a strong need for efficient subdata selection methods that support reliable statistical inference while using limited computational resources. Before selecting subdata, it can be beneficial to carry out variable selection, since only a small subset of the many available variables is typically relevant. A method designed for settings in which both the dataset size and the number of variables are large is proposed. The method first identifies the active variables using a procedure inspired by random LASSO, and then selects subdata through leverage scores to construct a predictive model. Compared with existing methods in the literature, including approaches based on the full dataset, the method achieves better performance in both variable selection and prediction, while also substantially reducing computation time. Its effectiveness is demonstrated through simulation studies and a real-data application.
