B1985
Title: Smart initialisation and approximate loss function for robust regression
Authors: Thomas Servotte - University of Antwerp (Belgium) [presenting]
Tim Verdonck - KU Leuven and UAntwerpen - imec (Belgium)
Jakob Raymaekers - KU Leuven (Belgium)
Abstract: Two of the most common methods for robust regression are least trimmed squares (LTS) and least median squares (LMS) regression. Both of these methods require sorting the squared residuals. Because sorting is not a differentiable operation, end-to-end optimalisation with gradient-based methods is not stable. Furthermore, existing algorithms for estimating LTS and LMS regressors rely on multiple random initial starting points. We propose and investigate two potential improvements to LTS and LMS: (1) the use of soft differentiable sorting in the loss functions and (2) deterministic initialisation of the estimators using the wrapping transformation. We show that deterministic initialisation has significant benefits for LTS and LMS, both for predictive accuracy and computational speed. The soft loss function mostly benefits LMS, as it makes it possible to apply iterative optimisation schemes to the LMS loss function. We also demonstrate the potential application of the Soft LTS loss function to non-linear regression problems using neural networks.
