  We investigate the problem of sequential linear data prediction for real life
big data applications. The second order algorithms, i.e., Newton-Raphson
Methods, asymptotically achieve the performance of the "best" possible linear
data predictor much faster compared to the first order algorithms, e.g., Online
Gradient Descent. However, implementation of these methods is not usually
feasible in big data applications because of the extremely high computational
needs. Regular implementation of the Newton-Raphson Methods requires a
computational complexity in the order of $O(M^2)$ for an $M$ dimensional
feature vector, while the first order algorithms need only $O(M)$. To this end,
in order to eliminate this gap, we introduce a highly efficient implementation
reducing the computational complexity of the Newton-Raphson Methods from
quadratic to linear scale. The presented algorithm provides the well-known
merits of the second order methods while offering the computational complexity
of $O(M)$. We utilize the shifted nature of the consecutive feature vectors and
do not rely on any statistical assumptions. Therefore, both regular and fast
implementations achieve the same performance in the sense of mean square error.
We demonstrate the computational efficiency of our algorithm on real life
sequential big datasets. We also illustrate that the presented algorithm is
numerically stable.
