  In this paper, energy efficient power allocation for the uplink of a
multi-cell massive MIMO system is investigated. With the simplified power
consumption model, the problem of power allocation is formulated as a
constrained Markov decision process (CMDP) framework with infinite-horizon
expected discounted total reward, which takes into account different quality of
service (QoS) requirements for each user terminal (UT). We propose an offline
solution containing the value iteration and Q-learning algorithms, which can
obtain the global optimum power allocation policy. Simulation results show that
our proposed policy performs very close to the ergodic optimal policy.
