  To reduce datacenter energy consumption and cost, current practice has
considered demand-proportional resource provisioning schemes, where servers are
turned on/off according to the load of requests.
  Most existing work considers instantaneous (Internet) requests only, which
are explicitly or implicitly assumed to be delay-sensitive. On the other hand,
in datacenters, there exist a vast amount of delay-tolerant jobs, such as
background/maintainance jobs. In this paper, we explicitly differentiate
delay-sensitive jobs and delay tolerant jobs. We focus on the problem of using
delay-tolerant jobs to fill the extra capacity of datacenters, referred to as
trough/valley filling. Giving a higher priority to delay-sensitive jobs, our
schemes complement to most existing demand-proportional resource provisioning
schemes. Our goal is to design intelligent trough filling mechanisms that are
energy efficient and also achieve good delay performance. Specifically, we
propose two joint dynamic speed scaling and traffic shifting schemes, one
subgradient-based and the other queue-based. Our schemes assume little
statistical information of the system, which is usually difficult to obtain in
practice. In both schemes, energy cost saving comes from dynamic speed scaling,
statistical multiplexing, electricity price diversity, and service efficiency
diversity. In addition, good delay performance is achieved in the queue-based
scheme via load shifting and capacity allocation based on queue conditions.
Practical issues that may arise in datacenter networks are considered,
including capacity and bandwidth constraint, service agility constraint, and
load shifting cost. We use both artificial and real datacenter traces to
evaluate the proposed schemes.
