  Resource allocation under uncertainty is a classical problem in city-scale
cyber-physical systems. Consider emergency response as an example; urban
planners and first responders optimize the location of ambulances to minimize
expected response times to incidents such as road accidents. Typically, such
problems deal with sequential decision-making under uncertainty and can be
modeled as Markov (or semi-Markov) decision processes. The goal of the
decision-maker is to learn a mapping from states to actions that can maximize
expected rewards. While online, offline, and decentralized approaches have been
proposed to tackle such problems, scalability remains a challenge for
real-world use-cases. We present a general approach to hierarchical planning
that leverages structure in city-level CPS problems for resource allocation. We
use emergency response as a case study and show how a large resource allocation
problem can be split into smaller problems. We then use Monte-Carlo planning
for solving the smaller problems and managing the interaction between them.
Finally, we use data from Nashville, Tennessee, a major metropolitan area in
the United States, to validate our approach. Our experiments show that the
proposed approach outperforms state-of-the-art approaches used in the field of
emergency response.
