  In this paper we present a scalable deep learning framework for finding
Markovian Nash Equilibria in multi-agent stochastic games using fictitious
play. The motivation is inspired by theoretical analysis of Forward Backward
Stochastic Differential Equations (FBSDE) and their implementation in a deep
learning setting, which is the source of our algorithm's sample efficiency
improvement. By taking advantage of the permutation-invariant property of
agents in symmetric games, the scalability and performance is further enhanced
significantly. We showcase superior performance of our framework over the
state-of-the-art deep fictitious play algorithm on an inter-bank
lending/borrowing problem in terms of multiple metrics. More importantly, our
approach scales up to 3000 agents in simulation, a scale which, to the best of
our knowledge, represents a new state-of-the-art. We also demonstrate the
applicability of our framework in robotics on a belief space autonomous racing
problem.
