  We study how to support elasticity, i.e., the ability to dynamically adjust
the parallelism (number of GPUs), for deep neural network (DNN) training.
Elasticity can benefit multi-tenant GPU cluster management in many ways, e.g.,
achieving various scheduling objectives (e.g., job throughput, job completion
time, GPU efficiency) according to cluster load variations, maximizing the use
of transient idle resources, performance profiling, job migration, and
straggler mitigation. However, existing parallelism adjustment strategies incur
high overheads, which hinder many applications from making effective use of
elasticity. We propose EDL to enable low-overhead elastic deep learning with a
simple API. We present techniques that are necessary to reduce the overhead of
parallelism adjustments, such as stop-free scaling and dynamic data pipeline.
We also demonstrate that EDL can indeed bring significant benefits to the
above-listed applications in GPU cluster management.
