  There is an increase in usage of smaller cells or femtocells to improve
performance and coverage of next-generation heterogeneous wireless networks
(HetNets). However, the interference caused by femtocells to neighboring cells
is a limiting performance factor in dense HetNets. This interference is being
managed via distributed resource allocation methods. However, as the density of
the network increases so does the complexity of such resource allocation
methods. Yet, unplanned deployment of femtocells requires an adaptable and
self-organizing algorithm to make HetNets viable. As such, we propose to use a
machine learning approach based on Q-learning to solve the resource allocation
problem in such complex networks. By defining each base station as an agent, a
cellular network is modelled as a multi-agent network. Subsequently,
cooperative Q-learning can be applied as an efficient approach to manage the
resources of a multi-agent network. Furthermore, the proposed approach
considers the quality of service (QoS) for each user and fairness in the
network. In comparison with prior work, the proposed approach can bring more
than a four-fold increase in the number of supported femtocells while using
cooperative Q-learning to reduce resource allocation overhead.
