  ML workloads are becoming increasingly popular in the cloud. Good cloud
training performance is contingent on efficient parameter exchange among VMs.
We find that Collectives, the widely used distributed communication algorithms,
cannot perform optimally out of the box due to the hierarchical topology of
datacenter networks and multi-tenancy nature of the cloudenvironment.In this
paper, we present Cloud Collectives , a prototype that accelerates collectives
by reordering theranks of participating VMs such that the communication pattern
dictated by the selected collectives operation best exploits the locality in
the network.Collectives is non-intrusive, requires no code changes nor rebuild
of an existing application, and runs without support from cloud providers. Our
preliminary application of Cloud Collectives on allreduce operations in public
clouds results in a speedup of up to 3.7x in multiple microbenchmarks and 1.3x
in real-world workloads of distributed training of deep neural networks and
gradient boosted decision trees using state-of-the-art frameworks.
