  With the advantages of Millimeter wave in wireless communication network, the
coverage radius and inter-site distance can be further reduced, the ultra dense
network (UDN) becomes the mainstream of future networks. The main challenge
faced by UDN is the serious inter-site interference, which needs to be
carefully addressed by joint user association and resource allocation methods.
In this paper, we propose a multi-agent Q-learning based method to jointly
optimize the user association and resource allocation in UDN. The deep
Q-network is applied to guarantee the convergence of the proposed method.
Simulation results reveal the effectiveness of the proposed method and
different performances under different simulation parameters are evaluated.
