Robust PCA vs. robust Mahalanobis distance for outlier detection

Robust PCA (as developed by Candes et al 2009 or better yet Netrepalli et al 2014) is a popular method for multivariate outlier detection, but Mahalanobis distance can also be used for outlier detection given a robust, regularized estimate of the covariance matrix. I’m curious about the (dis)advantages of using one method over the other.

My intuition tells me that the greatest distinction between the two is the following: When the data set is “small” (in a statistical sense), robust PCA will give a lower-rank covariance while robust covariance matrix estimation will instead give a full-rank covariance due to the Ledoit-Wolf regularization. How does this in turn affect outlier detection?

Answer

This paper compares some methods in this area. They refer to the Robust PCA approach you linked to as “PCP” (principal components pursuit) and the family of methods you linked to for robust covariance estimation as M-estimators.

They argue that

PCP is designed for uniformly corrupted coordinates of data, instead of corrupted data points (i.e., outliers), therefore, the comparison with PCP is somewhat unfair for this kind of data

and show that PCP (aka robust PCA) can fail for outlier detection in some cases.

They also talk about three kinds of “enemies of subspace recovery,” i.e. different kinds of outliers, and which kinds of methods might do well for dealing with each one. Comparing your own outliers with the three kinds of “enemies” discussed here might help you pick an approach.

Attribution
Source : Link , Question Author : Mustafa Eisa , Answer Author : David J. Harris

Leave a Comment