  Though the method of least squares has been used for a long time in solving
signal processing problems, in the recent field of sparse recovery from
compressed measurements, this method has not been given much attention. In this
paper we show that a method in the least squares family, known in the
literature as Orthogonal Least Squares (OLS), adapted for compressed recovery
problems, has competitive recovery performance and computation complexity, that
makes it a suitable alternative to popular greedy methods like Orthogonal
Matching Pursuit (OMP). We show that with a slight modification, OLS can
exactly recover a $K$-sparse signal, embedded in an $N$ dimensional space
($K<<N$) in $M=\mathcal{O}(K\log (N/K))$ no of measurements with Gaussian
dictionaries. We also show that OLS can be easily implemented in such a way
that it requires $\mathcal{O}(KMN)$ no of floating point operations similar to
that of OMP. In this paper performance of OLS is also studied with sensing
matrices with correlated dictionary, in which algorithms like OMP does not
exhibit good recovery performance. We study the recovery performance of OLS in
a specific dictionary called \emph{generalized hybrid dictionary}, which is
shown to be a correlated dictionary, and show numerically that OLS has is far
superior to OMP in these kind of dictionaries in terms of recovery performance.
Finally we provide analytical justifications that corroborate the findings in
the numerical illustrations.
