  Power management in multi-server data centers~especially at scale is a vital
issue of increasing importance in cloud computing paradigm. Existing studies
mostly consider thresholds on the number of idle servers to switch the servers
on or off and suffer from scalability issues. As a natural approach in
view~of~the Markovian assumption, we present a multi-level continuous-time
Markov decision process (CTMDP) model based on state aggregation of
multi-server data centers with setup times that interestingly overcomes the
inherent intractability of traditional MDP approaches due to their colossal
state-action space. The beauty of the presented model is that, while it keeps
loyalty to the Markovian behavior, it approximates the calculation of the
transition probabilities in a way that keeps the accuracy of the results at a
desirable level. Moreover, near-optimal performance is attained at the expense
of the increased state-space dimensionality by tuning the number of levels in
the multi-level approach. The simulation results were promising and confirm
that in many scenarios of interest, the proposed approach attains noticeable
improvements, namely a near 50% reduction in the size of CTMDP while yielding
better rewards as compared to existing fixed threshold-based policies and
aggregation methods.
