  We review existing approaches to mathematical modeling and analysis of
multi-agent systems in which complex collective behavior arises out of local
interactions between many simple agents. Though the behavior of an individual
agent can be considered to be stochastic and unpredictable, the collective
behavior of such systems can have a simple probabilistic description. We show
that a class of mathematical models that describe the dynamics of collective
behavior of multi-agent systems can be written down from the details of the
individual agent controller. The models are valid for Markov or memoryless
agents, in which each agents future state depends only on its present state and
not any of the past states. We illustrate the approach by analyzing in detail
applications from the robotics domain: collaboration and foraging in groups of
robots.
