  One of the goals in scaling sequential machine learning methods pertains to
dealing with high-dimensional data spaces. A key related challenge is that many
methods heavily depend on obtaining the inverse covariance matrix of the data.
It is well known that covariance matrix estimation is problematic when the
number of observations is relatively small compared to the number of variables.
A common way to tackle this problem is through the use of a shrinkage estimator
that offers a compromise between the sample covariance matrix and a
well-conditioned matrix, with the aim of minimizing the mean-squared error. We
derived sequential update rules to approximate the inverse shrinkage estimator
of the covariance matrix. The approach paves the way for improved large-scale
machine learning methods that involve sequential updates.
