  Over the past ten years, many different approaches have been proposed for
different aspects of the problem of resources management for long running,
dynamic and diverse workloads such as processing query streams or distributed
deep learning. Particularly for applications consisting of containerized
microservices, researchers have attempted to address problems of dynamic
selection of, for example: types and quantities of virtualized services (e.g.,
IaaS/VMs), vertical and horizontal scaling of different microservices,
assigning microservices to VMs, task scheduling, or some combination thereof.
In this context, we argue that frameworks like simulated annealing are highly
suitable for online navigation of trade-offs between performance (SLO) and
cost, particularly when the complex workloads and cloud-service offerings vary
over time. Based on a macroscopic objective that combines both performance and
cost terms, annealing facilitates light-weight and coherent policies of
exploration and exploitation. In this paper, we first give some background on
simulated annealing and then experimentally demonstrate its usefulness for
different case studies, including service selection for both a single type of
workload (e.g., distributed deep learning) and a mixture of workload types
(exploring a partially categorical set of options), and container sizing for
microservice benchmarks. We conclude with a discussion of how the basic
annealing platform can be applied to other resource-management problems,
hybridized with other methods, and accommodate user-specified rules of thumb.
