  In the course of the last century, Principal Component Analysis (PCA) have
become one of the pillars of modern scientific methods. Although PCA is
normally addressed as a statistical tool aiming at finding orthogonal
directions on which the variance is maximized, its first introduction by
Pearson at 1901 was done through defining a non-linear least-squares
minimization problem of fitting a plane to scattered data points. Thus, it
seems natural that PCA and linear least-squares regression are somewhat
related, as they both aim at fitting planes to data points. In this paper, we
present a connection between the two approaches. Specifically, we present an
iterated linear least-squares approach, yielding a sequence of subspaces, which
converges to the space spanned by the leading principal components (i.e.,
principal space).
