  Zero-sum stochastic games have found important applications in a variety of
fields, from machine learning to economics. Work on this model has primarily
focused on the computation of Nash equilibrium due to its effectiveness in
solving adversarial board and video games. Unfortunately, a Nash equilibrium is
not guaranteed to exist in zero-sum stochastic games when the payoffs at each
state are not convex-concave in the players' actions. A Stackelberg
equilibrium, however, is guaranteed to exist. Consequently, in this paper, we
study zero-sum stochastic Stackelberg games. Going beyond known existence
results for (non-stationary) Stackelberg equilibria, we prove the existence of
recursive (i.e., Markov perfect) Stackelberg equilibria (recSE) in these games,
provide necessary and sufficient conditions for a policy profile to be a recSE,
and show that recSE can be computed in (weakly) polynomial time via value
iteration. Finally, we show that zero-sum stochastic Stackelberg games can
model the problem of pricing and allocating goods across agents and time. More
specifically, we propose a zero-sum stochastic Stackelberg game whose recSE
correspond to the recursive competitive equilibria of a large class of
stochastic Fisher markets. We close with a series of experiments that showcase
how our methodology can be used to solve the consumption-savings problem in
stochastic Fisher markets.
