  Kaczmarz method is one popular iterative method for solving inverse problems,
especially in computed tomography. Recently, it was established that a
randomized version of the method enjoys an exponential convergence for
well-posed problems, and the convergence rate is determined by a variant of the
condition number. In this work, we analyze the preasymptotic convergence
behavior of the randomized Kaczmarz method, and show that the low-frequency
error (with respect to the right singular vectors) decays faster during first
iterations than the high-frequency error. Under the assumption that the inverse
solution is smooth (e.g., sourcewise representation), the result explains the
fast empirical convergence behavior, thereby shedding new insights into the
excellent performance of the randomized Kaczmarz method in practice. Further,
we propose a simple strategy to stabilize the asymptotic convergence of the
iteration by means of variance reduction. We provide extensive numerical
experiments to confirm the analysis and to elucidate the behavior of the
algorithms.
