  We establish an improved classical algorithm for solving linear systems in a
model analogous to the QRAM that is used by quantum linear solvers. Precisely,
for the linear system $A\x = \b$, we show that there is a classical algorithm
that outputs a data structure for $\x$ allowing sampling and querying to the
entries, where $\x$ is such that $\|\x - A^{+}\b\|\leq \epsilon \|A^{+}\b\|$.
This output can be viewed as a classical analogue to the output of quantum
linear solvers. The complexity of our algorithm is $\widetilde{O}(\kappa_F^4
\kappa^2/\epsilon^2 )$, where $\kappa_F = \|A\|_F\|A^{+}\|$ and $\kappa =
\|A\|\|A^{+}\|$. This improves the previous best algorithm [Gily{\'e}n, Song
and Tang, arXiv:2009.07268] of complexity $\widetilde{O}(\kappa_F^6
\kappa^6/\epsilon^4)$. Our algorithm is based on the randomized Kaczmarz
method, which is a particular case of stochastic gradient descent. We also find
that when $A$ is row sparse, this method already returns an approximate
solution $\x$ in time $\widetilde{O}(\kappa_F^2)$, while the best quantum
algorithm known returns $\ket{\x}$ in time $\widetilde{O}(\kappa_F)$ when $A$
is stored in the QRAM data structure. As a result, assuming access to QRAM and
if $A$ is row sparse, the speedup based on current quantum algorithms is
quadratic.
