  In this paper, we aim at interference mitigation in 5G millimeter-Wave
(mm-Wave) communications by employing beamforming and Non-Orthogonal Multiple
Access (NOMA) techniques with the aim of improving network's aggregate rate.
Despite the potential capacity gains of mm-Wave and NOMA, many technical
challenges might hinder that performance gain. In particular, the performance
of Successive Interference Cancellation (SIC) diminishes rapidly as the number
of users increases per beam, which leads to higher intra-beam interference.
Furthermore, intersection regions between adjacent cells give rise to
inter-beam inter-cell interference. To mitigate both interference levels,
optimal selection of the number of beams in addition to best allocation of
users to those beams is essential. In this paper, we address the problem of
joint user-cell association and selection of number of beams for the purpose of
maximizing the aggregate network capacity. We propose three machine
learning-based algorithms; transfer Q-learning (TQL), Q-learning, and Best SINR
association with Density-based Spatial Clustering of Applications with Noise
(BSDC) algorithms and compare their performance under different scenarios.
Under mobility, TQL and Q-learning demonstrate 12% rate improvement over BSDC
at the highest offered traffic load. For stationary scenarios, Q-learning and
BSDC outperform TQL, however TQL achieves about 29% convergence speedup
compared to Q-learning.
