  We consider the problem of decentralized optimization in networks with
communication delays. To accommodate delays, we need decentralized optimization
algorithms that work on directed graphs. Existing approaches require nodes to
know their out-degree to achieve convergence. We propose a novel gossip-based
algorithm that circumvents this requirement, allowing decentralized
optimization in networks with communication delays. We prove that our algorithm
converges on non-convex objectives, with the same main complexity order term as
centralized Stochastic Gradient Descent (SGD), and show that the graph topology
and the delays only affect the higher order terms. We provide numerical
simulations that illustrate our theoretical results.
