  This paper considers the distributed learning problem where a group of agents
cooperatively minimizes the summation of their local cost functions based on
peer-to-peer communication. Particularly, we propose a highly efficient
algorithm, termed ``B-ary Tree Push-Pull'' (BTPP), that employs two B-ary
spanning trees for distributing the information related to the parameters and
stochastic gradients across the network. The simple method is efficient in
communication since each agent interacts with at most $(B+1)$ neighbors per
iteration. More importantly, BTPP achieves linear speedup for smooth nonconvex
objective functions with only $\tilde{O}(n)$ transient iterations,
significantly outperforming the state-of-the-art results to the best of our
knowledge.
