  Nonzero sum games typically have multiple Nash equilibriums (or no
equilibrium), and unlike the zero sum case, they may have different values at
different equilibriums. Instead of focusing on the existence of individual
equilibriums, we study the set of values over all equilibriums, which we call
the set value of the game. The set value is unique by nature and always exists
(with possible value $\emptyset$). Similar to the standard value function in
control literature, it enjoys many nice properties such as regularity,
stability, and more importantly the dynamic programming principle. There are
two main features in order to obtain the dynamic programming principle: (i) we
must use closed-loop controls (instead of open-loop controls); (ii) we must
allow for path dependent controls, even if the problem is in a state dependent
(Markovian) setting. We shall consider both discrete and continuous time models
with finite time horizon. For the latter we will also provide a duality
approach through certain standard PDE (or path dependent PDE), which is quite
efficient for numerically computing the set value of the game.
