Title: Learning from MOM's principle
Authors: Guillaume Lecue - CNRS and ENSAE (France) [presenting]
Matthieu Lerasle - CNRS (France)
Abstract: This is the second part of a session on a mini-maximization Median of Means estimator. In this second part, we present several algorithms to compute our estimator and discuss practical performances in various settings. Several points of interest will be stressed: 1) many algorithms designed for the ERM and its regularized versions can easily be adapted to compute the MOMs versions of these estimators (we will present in details an alternating ADMM algorithm designed for the MOM estimator); 2) these MOMs versions inherit robustness properties of MOMs estimators, in particular they are naturally resistant to outliers, heavy-tailed and non identically distributed data; 3) when applied to computationally heavy non-linear methods, MOMs version, operating on smaller batch of data, are much faster to implement. For some large distributed data-sets, the MOM estimator is tractable on a personal computer while the original version requires much stronger computational powers; 4) compared to other robust estimators, MOMs estimators can naturally be fixed to avoid local maxima; 5) we also derive a concept of outlier importance for the problem of outliers detection is naturally associated to a notion of centrality of data. Illustrations of those points will be outlined in particular by a MOM version of the classical LASSO.