  Millimeter-wave (mmWave) communication is anticipated to provide significant
throughout gains in urban scenarios. To this end, network densification is a
necessity to meet the high traffic volume generated by smart phones, tablets,
and sensory devices while overcoming large pathloss and high blockages at
mmWaves frequencies. These denser networks are created with users deploying
small mmWave base stations (BSs) in a plug-and-play fashion. Although, this
deployment method provides the required density, the amorphous deployment of
BSs needs distributed management. To address this difficulty, we propose a
self-organizing method to allocate power to mmWave BSs in an ultra dense
network. The proposed method consists of two parts: clustering using fast local
clustering and power allocation via Q-learning. The important features of the
proposed method are its scalability and self-organizing capabilities, which are
both important features of 5G. Our simulations demonstrate that the introduced
method, provides required quality of service (QoS) for all the users
independent of the size of the network.
