Workshop SDS: Registration
View Submission - SDS2022
Title: Statistical guarantees for variational automatic relevance determination Authors:  Feng Liang - University of Illinois at Urbana-Champaign (United States) [presenting]
Zihe Liu - University of Illinois at Urbana-Champaign (United States)
Abstract: The Automatic Relevance Determination (ARD) model is studied for high-dimensional linear regression under sparsity constraints. For each regression parameter, the ARD prior places a Gaussian distribution with mean zero, and variance is a hyper-parameter that needs to be learned from the data. We focus on a variational procedure, which approximates the posterior distribution by independent Gaussian distributions, one for each parameter. It can be shown that for some parameters the corresponding Gaussian distribution will degenerate to a point mass at zero; that is, some variables will be automatically filtered out by the ARD procedure. Although ARD and this variational framework have been studied before, little is known about the theoretical properties of the variational solution. The main contribution is to establish convergence results, in terms of parameter estimation and variable selection, for the variational solution of an ARD model.