  The designs of many large-scale systems today, from traffic routing
environments to smart grids, rely on game-theoretic equilibrium concepts.
However, as the size of an $N$-player game typically grows exponentially with
$N$, standard game theoretic analysis becomes effectively infeasible beyond a
low number of players. Recent approaches have gone around this limitation by
instead considering Mean-Field games, an approximation of anonymous $N$-player
games, where the number of players is infinite and the population's state
distribution, instead of every individual player's state, is the object of
interest. The practical computability of Mean-Field Nash equilibria, the most
studied Mean-Field equilibrium to date, however, typically depends on
beneficial non-generic structural properties such as monotonicity or
contraction properties, which are required for known algorithms to converge. In
this work, we provide an alternative route for studying Mean-Field games, by
developing the concepts of Mean-Field correlated and coarse-correlated
equilibria. We show that they can be efficiently learnt in \emph{all games},
without requiring any additional assumption on the structure of the game, using
three classical algorithms. Furthermore, we establish correspondences between
our notions and those already present in the literature, derive optimality
bounds for the Mean-Field - $N$-player transition, and empirically demonstrate
the convergence of these algorithms on simple games.
