  In decentralized optimization over networks, each node in the network has a
portion of the global objective function and the aim is to collectively
optimize this function. Gradient tracking methods have emerged as a popular
alternative for solving such problems due to their strong theoretical
guarantees and robust empirical performance. These methods perform two
operations (steps) at each iteration: (1) compute local gradients at each node,
and (2) communicate local information with neighboring nodes. The complexity of
these two steps can vary significantly across applications. In this work, we
present a flexible gradient tracking algorithmic framework designed to balance
the composition of communication and computation steps over the optimization
process using a randomized scheme. The proposed framework is general, unifies
gradient tracking methods, and recovers classical gradient tracking methods as
special cases. We establish convergence guarantees in expectation and
illustrate how the complexity of communication and computation steps can be
balanced using the provided flexibility. Finally, we illustrate the performance
of the proposed methods on quadratic and logistic regression problems, and
compare against popular algorithms from the literature.
