  Big data, including applications with high security requirements, are often
collected and stored on multiple heterogeneous devices, such as mobile devices,
drones and vehicles. Due to the limitations of communication costs and security
requirements, it is of paramount importance to extract information in a
decentralized manner instead of aggregating data to a fusion center. To train
large-scale machine learning models, edge/fog computing is often leveraged as
an alternative to centralized learning. We consider the problem of learning
model parameters in a multi-agent system with data locally processed via
distributed edge nodes. A class of mini-batch stochastic alternating direction
method of multipliers (ADMM) algorithms is explored to develop the distributed
learning model. To address two main critical challenges in distributed
networks, i.e., communication bottleneck and straggler nodes (nodes with slow
responses), error-control-coding based stochastic incremental ADMM is
investigated. Given an appropriate mini-batch size, we show that the mini-batch
stochastic ADMM based method converges in a rate of $O(\frac{1}{\sqrt{k}})$,
where $k$ denotes the number of iterations. Through numerical experiments, it
is revealed that the proposed algorithm is communication-efficient, rapidly
responding and robust in the presence of straggler nodes compared with state of
the art algorithms.
