CMStatistics 2022: Start Registration
View Submission - CMStatistics
Title: A novel differentiable unification of least absolute deviations and least squares Authors:  Kevin Burke - University of Limerick (Ireland) [presenting]
Abstract: Arguably, two of the most important error distributions are the normal and the Laplace distributions, respectively, being equivalent to classical least squares and least absolute deviations estimation. The key difference between these procedures is the use of a square function or an absolute value function within the objective function. Although least absolute deviations produce regression coefficients that are less impacted by outliers, their usage in applications has historically been much less widespread than least squares. This is likely due to the non-differentiable nature of the objective function, which requires more specialized treatment. However, we demonstrate that standard gradient-based optimization procedures can be applied if the absolute value function is replaced with a commonly used smooth approximation. Perhaps unexpectedly, we also show that this same procedure (designed for least absolute deviations approximation) can yield least squares estimates for particular values of its smoothing parameter. Ultimately, we develop a unified likelihood-based estimation procedure that can produce both the least absolute deviations and least squares estimates, as well as solutions between the two. Moreover, due to the equivalence with Laplace and normal distributions, we derive a new Laplace-normal-type distribution whose density function is differentiable.