  This paper is triggered by the preprint "\emph{Computing Matrix Squareroot
via Non Convex Local Search}" by Jain et al.
(\textit{\textcolor{blue}{arXiv:1507.05854}}), which analyzes gradient-descent
for computing the square root of a positive definite matrix. Contrary to claims
of~\citet{jain2015}, our experiments reveal that Newton-like methods compute
matrix square roots rapidly and reliably, even for highly ill-conditioned
matrices and without requiring commutativity. We observe that gradient-descent
converges very slowly primarily due to tiny step-sizes and ill-conditioning. We
derive an alternative first-order method based on geodesic convexity: our
method admits a transparent convergence analysis ($< 1$ page), attains linear
rate, and displays reliable convergence even for rank deficient problems.
Though superior to gradient-descent, ultimately our method is also outperformed
by a well-known scaled Newton method. Nevertheless, the primary value of our
work is its conceptual value: it shows that for deriving gradient based methods
for the matrix square root, \emph{the manifold geometric view of positive
definite matrices can be much more advantageous than the Euclidean view}.
