  In this paper we study stochastic dynamic games with many players; these are
a fundamental model for a wide range of economic applications. The standard
solution concept for such games is Markov perfect equilibrium (MPE), but it is
well known that MPE computation becomes intractable as the number of players
increases. We instead consider the notion of stationary equilibrium (SE), where
players optimize assuming the empirical distribution of others' states remains
constant at its long run average. We make two main contributions. First, we
provide a rigorous justification for using SE. In particular, we provide a
parsimonious collection of exogenous conditions over model primitives that
guarantee existence of SE, and ensure that an appropriate approximation
property to MPE holds, in a general model with possibly unbounded state spaces.
Second, we draw a significant connection between the validity of SE, and market
structure: under the same conditions that imply SE exist and approximates MPE
well, the market becomes fragmented in the limit of many firms. To illustrate
this connection, we study in detail a series of dynamic oligopoly examples.
These examples show that our conditions enforce a form of "decreasing returns
to larger states"; this yields fragmented industries in the limit. By contrast,
violation of these conditions suggests "increasing returns to larger states"
and potential market concentration. In that sense, our work uses a fully
dynamic framework to also contribute to a longstanding issue in industrial
organization: understanding the determinants of market structure in different
industries.
