  In the era of big data, one of the key challenges is the development of novel
optimization algorithms that can accommodate vast amounts of data while at the
same time satisfying constraints and limitations of the problem under study.
The need to solve optimization problems is ubiquitous in essentially all
quantitative areas of human endeavor, including industry and science. In the
last decade there has been a surge in the demand from practitioners, in fields
such as machine learning, computer vision, artificial intelligence, signal
processing and data science, for new methods able to cope with these new large
scale problems.
  In this thesis we are focusing on the design, complexity analysis and
efficient implementations of such algorithms. In particular, we are interested
in the development of randomized iterative methods for solving large scale
linear systems, stochastic quadratic optimization problems, the best
approximation problem and quadratic optimization problems. A large part of the
thesis is also devoted to the development of efficient methods for obtaining
average consensus on large scale networks.
