  In this paper, we apply the idea of fictitious play to design deep neural
networks (DNNs), and develop deep learning theory and algorithms for computing
the Nash equilibrium of asymmetric $N$-player non-zero-sum stochastic
differential games, for which we refer as \emph{deep fictitious play}, a
multi-stage learning process. Specifically at each stage, we propose the
strategy of letting individual player optimize her own payoff subject to the
other players' previous actions, equivalent to solve $N$ decoupled stochastic
control optimization problems, which are approximated by DNNs. Therefore, the
fictitious play strategy leads to a structure consisting of $N$ DNNs, which
only communicate at the end of each stage. The resulted deep learning algorithm
based on fictitious play is scalable, parallel and model-free, {\it i.e.},
using GPU parallelization, it can be applied to any $N$-player stochastic
differential game with different symmetries and heterogeneities ({\it e.g.},
existence of major players). We illustrate the performance of the deep learning
algorithm by comparing to the closed-form solution of the linear quadratic
game. Moreover, we prove the convergence of fictitious play under appropriate
assumptions, and verify that the convergent limit forms an open-loop Nash
equilibrium. We also discuss the extensions to other strategies designed upon
fictitious play and closed-loop Nash equilibrium in the end.
