COMPSTAT 2016: Start Registration
View Submission - COMPSTAT
Title: Detecting anomalous data cells Authors:  Peter Rousseeuw - KU Leuven (Belgium) [presenting]
Wannes Van den Bossche - KU Leuven (Belgium)
Abstract: A multivariate dataset consists of $n$ cases in $d$ dimensions, and is often stored in an $n$ by $d$ data matrix. It is well-known that real data may contain outliers. Depending on the situation, outliers may be (a) undesirable errors which can adversely affect the data analysis, or (b) valuable nuggets of unexpected information. In statistics and data analysis the word outlier usually refers to a row of the data matrix, and the methods to detect such outliers only work when at least half the rows are clean. But often many rows have a few contaminated cell values, which may not be visible by looking at each variable (column) separately. A method to detect anomalous data cells is proposed, which takes the correlations between the variables into account. It has no restriction on the number of clean rows, and can deal with high dimensions. Other advantages are that it provides estimates of the `expected' values of the outlying cells, while imputing missing values at the same time. The method is illustrated on several real data sets, where it uncovers more structure than found by purely columnwise methods or purely rowwise methods. It can also serve as an initial step for estimating multivariate location and scatter matrices.