  Stochastic network optimization problems entail finding resource allocation
policies that are optimum on an average but must be designed in an online
fashion. Such problems are ubiquitous in communication networks, where
resources such as energy and bandwidth are divided among nodes to satisfy
certain long-term objectives. This paper proposes an asynchronous incremental
dual decent resource allocation algorithm that utilizes delayed stochastic
{gradients} for carrying out its updates. The proposed algorithm is well-suited
to heterogeneous networks as it allows the computationally-challenged or
energy-starved nodes to, at times, postpone the updates. The asymptotic
analysis of the proposed algorithm is carried out, establishing dual
convergence under both, constant and diminishing step sizes. It is also shown
that with constant step size, the proposed resource allocation policy is
asymptotically near-optimal. An application involving multi-cell coordinated
beamforming is detailed, demonstrating the usefulness of the proposed
algorithm.
