  Distributed machine learning (DML) can be an important capability for modern
military to take advantage of data and devices distributed at multiple vantage
points to adapt and learn. The existing distributed machine learning
frameworks, however, cannot realize the full benefits of DML, because they are
all based on the simple linear aggregation framework, but linear aggregation
cannot handle the $\textit{divergence challenges}$ arising in military
settings: the learning data at different devices can be heterogeneous
($\textit{i.e.}$, Non-IID data), leading to model divergence, but the ability
for devices to communicate is substantially limited ($\textit{i.e.}$, weak
connectivity due to sparse and dynamic communications), reducing the ability
for devices to reconcile model divergence. In this paper, we introduce a novel
DML framework called aggregation in the mirror space (AIMS) that allows a DML
system to introduce a general mirror function to map a model into a mirror
space to conduct aggregation and gradient descent. Adapting the convexity of
the mirror function according to the divergence force, AIMS allows automatic
optimization of DML. We conduct both rigorous analysis and extensive
experimental evaluations to demonstrate the benefits of AIMS. For example, we
prove that AIMS achieves a loss of
$O\left((\frac{m^{r+1}}{T})^{\frac1r}\right)$ after $T$ network-wide updates,
where $m$ is the number of devices and $r$ the convexity of the mirror
function, with existing linear aggregation frameworks being a special case with
$r=2$. Our experimental evaluations using EMANE (Extendable Mobile Ad-hoc
Network Emulator) for military communications settings show similar results:
AIMS can improve DML convergence rate by up to 57\% and scale well to more
devices with weak connectivity, all with little additional computation overhead
compared to traditional linear aggregation.
