  There are variety of computational algorithms need sequential sweeping;
sweeping based on specific order; on a structured grid, e.g., preconditioning
(smoothing) by SOR or ILU methods and solution of eikonal equation by fast
sweeping algorithm. Due to sequential nature, parallel implementation of these
algorithms usually leads to miss of efficiency; e.g. a significant convergence
rate decay. Therefore, there is an interest to parallelize sequential sweeping
procedures, keeping the efficiency of the original method simultaneously. This
paper goals to parallelize sequential sweeping algorithms on structured grids,
with emphasis on SOR and ILU preconditioners. The presented method can be
accounted as an overlapping domain decomposition method combined to a
multi-frontal sweeping procedure. The implementation of method in one and two
dimensions are discussed in details. The extension to higher dimensions and
general structured n-diagonal matrices is outlined. Introducing notion of
alternatively block upper-lower triangular matrices, the convergence theory is
established in general cases. Numerical results on model problems show that,
unlike related alternative parallel methods, the convergence rate and
efficiency of the presented method is close to the original sequential method.
Numerical results also support successful use of the presented method as a
cache efficient solver in sequential computations as well.
