  Large-scale distributed training in production datacenters constitutes a
challenging workload bottlenecked by network communication. In response, both
major industry players (e.g., Ultra Ethernet Consortium) and parts of academia
have surprisingly, and almost unanimously, agreed that packet spraying is
necessary to improve the performance of large-scale distributed training
workloads.
  In this paper, we challenge this prevailing belief and pose the question: How
close can a singlepath transport approach an optimal multipath transport? We
demonstrate that singlepath transport (from a NIC's perspective) is sufficient
and can perform nearly as well as an ideal multipath transport with packet
spraying, particularly in the context of distributed training in leaf-spine
topologies. Our assertion is based on four key observations about workloads
driven by collective communication patterns: (i) flows within a collective
start almost simultaneously, (ii) flow sizes are nearly equal, (iii) the
completion time of a collective is more crucial than individual flow completion
times, and (iv) flows can be split upon arrival. We analytically prove that
singlepath transport, using minimal flow splitting (at the application layer),
is equivalent to an ideal multipath transport with packet spraying in terms of
maximum congestion. Our preliminary evaluations support our claims. This paper
suggests an alternative agenda for developing next-generation transport
protocols tailored for large-scale distributed training.
