  In this paper, we propose a novel architecture for a deep learning system,
named k-degree layer-wise network, to realize efficient geo-distributed
computing between Cloud and Internet of Things (IoT). The geo-distributed
computing extends Cloud to the geographical verge of the network in the
neighbor of IoT. The basic ideas of the proposal include a k-degree constraint
and a layer-wise constraint. The k-degree constraint is defined such that the
degree of each vertex on the h-th layer is exactly k(h) to extend the existing
deep belief networks and control the communication cost. The layer-wise
constraint is defined such that the layer-wise degrees are monotonically
decreasing in positive direction to gradually reduce the dimension of data. We
prove the k-degree layer-wise network is sparse, while a typical deep neural
network is dense. In an evaluation on the M-distributed MNIST database, the
proposal is superior to a state-of-the-art model in terms of communication cost
and learning time with scalability.
