  In this paper, a novel approach for joint power control and user scheduling
is proposed for optimizing energy efficiency (EE), in terms of bits per unit
power, in ultra dense small cell networks (UDNs). To address this problem, a
dynamic stochastic game (DSG) is formulated between small cell base stations
(SBSs). This game enables to capture the dynamics of both queues and channel
states of the system. To solve this game, assuming a large homogeneous UDN
deployment, the problem is cast as a mean field game (MFG) in which the MFG
equilibrium is analyzed with the aid of two low-complexity tractable partial
differential equations. User scheduling is formulated as a stochastic
optimization problem and solved using the drift plus penalty (DPP) approach in
the framework of Lyapunov optimization. Remarkably, it is shown that by weaving
notions from Lyapunov optimization and mean field theory, the proposed solution
yields an equilibrium control policy per SBS which maximizes the network
utility while ensuring users' quality-of-service. Simulation results show that
the proposed approach achieves up to 18:1% gains in EE and 98.2% reductions in
the network's outage probability compared to a baseline model.
