  The canonical problem of solving a system of linear equations arises in
numerous contexts in information theory, communication theory, and related
fields. In this contribution, we develop a solution based upon Gaussian belief
propagation (GaBP) that does not involve direct matrix inversion. The iterative
nature of our approach allows for a distributed message-passing implementation
of the solution algorithm. In the first part of this thesis, we address the
properties of the GaBP solver. We characterize the rate of convergence, enhance
its message-passing efficiency by introducing a broadcast version, discuss its
relation to classical solution methods including numerical examples. We present
a new method for forcing the GaBP algorithm to converge to the correct solution
for arbitrary column dependent matrices.
  In the second part we give five applications to illustrate the applicability
of the GaBP algorithm to very large computer networks: Peer-to-Peer rating,
linear detection, distributed computation of support vector regression,
efficient computation of Kalman filter and distributed linear programming.
Using extensive simulations on up to 1,024 CPUs in parallel using IBM Bluegene
supercomputer we demonstrate the attractiveness and applicability of the GaBP
algorithm, using real network topologies with up to millions of nodes and
hundreds of millions of communication links. We further relate to several other
algorithms and explore their connection to the GaBP algorithm.
