  The paper is concerned with distributed learning and optimization in
large-scale settings. The well-known Fictitious Play (FP) algorithm has been
shown to achieve Nash equilibrium learning in certain classes of multi-agent
games. However, FP can be computationally difficult to implement when the
number of players is large. Sampled FP is a variant of FP that mitigates the
computational difficulties arising in FP by using a Monte-Carlo (i.e.,
sampling-based) approach. The Sampled FP algorithm has been studied both as a
tool for distributed learning and as an optimization heuristic for large-scale
problems. Despite its computational advantages, a shortcoming of Sampled FP is
that the number of samples that must be drawn in each round of the algorithm
grows without bound (on the order of $\sqrt{t}$, where $t$ is the round of the
repeated play). In this paper we propose Computationally Efficient Sampled FP
(CESFP)---a variant of Sampled FP in which only one sample need be drawn each
round of the algorithm (a substantial reduction from $O(\sqrt{t})$ samples per
round, as required in Sampled FP). CESFP operates using a
stochastic-approximation type rule to estimate the expected utility from round
to round. It is proven that the CESFP algorithm achieves Nash equilibrium
learning in the same sense as classical FP and Sampled FP. Simulation results
suggest that the convergence rate of CESFP (in terms of repeated-play
iterations) is similar to that of Sampled FP.
