  The idea of Innovation Search, which was initially proposed for data
clustering, was recently used for outlier detection. In the application of
Innovation Search for outlier detection, the directions of innovation were
utilized to measure the innovation of the data points. We study the Innovation
Values computed by the Innovation Search algorithm under a quadratic cost
function and it is proved that Innovation Values with the new cost function are
equivalent to Leverage Scores. This interesting connection is utilized to
establish several theoretical guarantees for a Leverage Score based robust PCA
method and to design a new robust PCA method. The theoretical results include
performance guarantees with different models for the distribution of outliers
and the distribution of inliers. In addition, we demonstrate the robustness of
the algorithms against the presence of noise. The numerical and theoretical
studies indicate that while the presented approach is fast and closed-form, it
can outperform most of the existing algorithms.
