  Distributed Deep Learning (DDL), as a paradigm, dictates the use of GPU-based
clusters as the optimal infrastructure for training large-scale Deep Neural
Networks (DNNs). However, the high cost of such resources makes them
inaccessible to many users. Public cloud services, particularly Spot Virtual
Machines (VMs), offer a cost-effective alternative, but their unpredictable
availability poses a significant challenge to the crucial checkpointing process
in DDL. To address this, we introduce DeepVM, a novel solution that recommends
cost-effective cluster configurations by intelligently balancing the use of
Spot and On-Demand VMs. DeepVM leverages a four-stage process that analyzes
instance performance using the FLOPP (FLoating-point Operations Per Price)
metric, performs architecture-level analysis with linear programming, and
identifies the optimal configuration for the user-specific needs. Extensive
simulations and real-world deployments in the AWS environment demonstrate that
DeepVM consistently outperforms other policies, reducing training costs and
overall makespan. By enabling cost-effective checkpointing with Spot VMs,
DeepVM opens up DDL to a wider range of users and facilitates a more efficient
training of complex DNNs.
