  Suppose $A \in \mathbb{R}^{n \times n}$ is invertible and we are looking for
the solution of $Ax = b$. Given an initial guess $x_1 \in \mathbb{R}$, we show
that by reflecting through hyperplanes generated by the rows of $A$, we can
generate an infinite sequence $(x_k)_{k=1}^{\infty}$ such that all elements
have the same distance to the solution, i.e. $\|x_k - x\| = \|x_1 - x\|$. If
the hyperplanes are chosen at random, averages over the sequence converge and
$$ \mathbb{E} \left\| x - \frac{1}{m} \sum_{k=1}^{m}{ x_k} \right\| \leq
\frac{1 + \|A\|_F \|A^{-1}\|}{\sqrt{m}} \cdot\|x-x_1\|.$$ The bound does not
depend on the dimension of the matrix. This introduces a purely geometric way
of attacking the problem: are there fast ways of estimating the location of the
center of a sphere from knowing many points on the sphere? Our convergence rate
(coinciding with that of the Random Kaczmarz method) comes from averaging, can
one do better?
