  Renewable energy sources such as wind and solar power have a high degree of
unpredictability and time-variation, which makes balancing demand and supply
challenging. One possible way to address this challenge is to harness the
inherent flexibility in demand of many types of loads. Introduced in this paper
is a technique for decentralized control for automated demand response that can
be used by grid operators as ancillary service for maintaining demand-supply
balance.
  A Markovian Decision Process (MDP) model is introduced for an individual
load. A randomized control architecture is proposed, motivated by the need for
decentralized decision making, and the need to avoid synchronization that can
lead to large and detrimental spikes in demand. An aggregate model for a large
number of loads is then developed by examining the mean field limit. A key
innovation is an LTI-system approximation of the aggregate nonlinear model,
with a scalar signal as the input and a measure of the aggregate demand as the
output. This makes the approximation particularly convenient for control design
at the grid level.
  The second half of the paper contains a detailed application of these results
to a network of residential pools. Simulations are provided to illustrate the
accuracy of the approximations and effectiveness of the proposed control
approach.
