Title: Bayesian ensemble of quantile regression trees
Authors: Mauro Bernardi - University of Padova (Italy) [presenting]
Paola Stolfi - Institute for applied mathematics Mauro Picone IAC (Italy)
Abstract: Decision trees and their population counterparts are becoming promising alternatives to classical linear regression techniques because of their superior ability to adapt to situations where the dependence structure between the response and the covariates is highly nonlinear. Despite their popularity, those methods have been developed for classification and regression, while often the conditional mean would not be enough when data strongly deviates from the Gaussian assumption. The proposed approach instead considers an ensemble of nonparametric regression trees to model the conditional quantile at level $\tau\in(0,1)$ of the response variable. Specifically, a flexible additive model is fitted to each partition of the data that corresponds to a given leaf of the tree. Unlike the most popular Bayesian approach (BART) that assumes a sum of regression trees, quantile estimates are obtained by averaging the ensemble trees, thereby reducing their variance. We develop a Bayesian procedure for fitting such models that effectively explores the space of B-spline functions of different orders that features the functional nonlinear relationship with the covariates. The approach is particularly valuable when skewness, fat-tails, outliers, truncated and censored data, and heteroskedasticity, can shadow the nature of the dependence between the variable of interest and the covariates.