  We study the synthesis of policies for multi-agent systems to implement
spatial-temporal tasks. We formalize the problem as a factored Markov decision
process subject to so-called graph temporal logic specifications. The
transition function and the spatial-temporal task of each agent depend on the
agent itself and its neighboring agents. The structure in the model and the
specifications enable to develop a distributed algorithm that, given a factored
Markov decision process and a graph temporal logic formula, decomposes the
synthesis problem into a set of smaller synthesis problems, one for each agent.
We prove that the algorithm runs in time linear in the total number of agents.
The size of the synthesis problem for each agent is exponential only in the
number of neighboring agents, which is typically much smaller than the number
of agents. We demonstrate the algorithm in case studies on disease control and
urban security. The numerical examples show that the algorithm can scale to
hundreds of agents.
