  Unmanned aerial vehicles (UAVs) are capable of serving as aerial base
stations (BSs) for providing both cost-effective and on-demand wireless
communications. This article investigates dynamic resource allocation of
multiple UAVs enabled communication networks with the goal of maximizing
long-term rewards. More particularly, each UAV communicates with a ground user
by automatically selecting its communicating users, power levels and
subchannels without any information exchange among UAVs. To model the
uncertainty of environments, we formulate the long-term resource allocation
problem as a stochastic game for maximizing the expected rewards, where each
UAV becomes a learning agent and each resource allocation solution corresponds
to an action taken by the UAVs. Afterwards, we develop a multi-agent
reinforcement learning (MARL) framework that each agent discovers its best
strategy according to its local observations using learning. More specifically,
we propose an agent-independent method, for which all agents conduct a decision
algorithm independently but share a common structure based on Q-learning.
Finally, simulation results reveal that: 1) appropriate parameters for
exploitation and exploration are capable of enhancing the performance of the
proposed MARL based resource allocation algorithm; 2) the proposed MARL
algorithm provides acceptable performance compared to the case with complete
information exchanges among UAVs. By doing so, it strikes a good tradeoff
between performance gains and information exchange overheads.
