  Iterative linear solvers have gained recent popularity due to their
computational efficiency and low memory footprint for large-scale linear
systems. The relaxation method, or Motzkin's method, can be viewed as an
iterative method that projects the current estimation onto the solution
hyperplane corresponding to the most violated constraint. Although this leads
to an optimal selection strategy for consistent systems, for inconsistent least
square problems, the strategy presents a tradeoff between convergence rate and
solution accuracy. We provide a theoretical analysis that shows Motzkin's
method offers an initially accelerated convergence rate and this acceleration
depends on the dynamic range of the residual. We quantify this acceleration for
Gaussian systems as a concrete example. Lastly, we include experimental
evidence on real and synthetic systems that support the analysis.
