  Cloud training platforms, such as Amazon Web Services and Huawei Cloud
provide users with computational resources to train their deep learning jobs.
Elastic training is a service embedded in cloud training platforms that
dynamically scales up or down the resources allocated to a job. The core
technique of an elastic training system is to best allocate limited resources
among heterogeneous jobs in terms of shorter queueing delay and higher training
efficiency. This paper presents an optimal resource allocator for elastic
training system that leverages a mixed-integer programming (MIP) model to
maximize the training progress of deep learning jobs. We take advantage of the
real-world job data obtained from ModelArts, the deep learning training
platform of Huawei Cloud and conduct simulation experiments to compare the
optimal resource allocator with a greedy one as benchmark. Numerical results
show that the proposed allocator can reduce queuing time by up to 32% and
accelerate training efficiency by up to 24% relative to the greedy resource
allocator, thereby greatly improving user experience with Huawei ModelArts and
potentially enabling the realization of higher profits for the product. Also,
the optimal resource allocator is fast in decision-making, taking merely 0.4
seconds on average.
