  Deploying ultra-dense networks that operate on millimeter wave (mmWave) band
is a promising way to address the tremendous growth on mobile data traffic.
However, one key challenge of ultra-dense mmWave network (UDmmN) is beam
management due to the high propagation delay, limited beam coverage as well as
numerous beams and users. In this paper, a novel systematic beam control scheme
is presented to tackle the beam management problem which is difficult due to
the nonconvex objective function. We employ double deep Q-network (DDQN) under
a federated learning (FL) framework to address the above optimization problem,
and thereby fulfilling adaptive and intelligent beam management in UDmmN. In
the proposed beam management scheme based on FL (BMFL), the non-rawdata
aggregation can theoretically protect user privacy while reducing handoff cost.
Moreover, we propose to adopt a data cleaning technique in the local model
training for BMFL, with the aim to further strengthen the privacy protection of
users while improving the learning convergence speed. Simulation results
demonstrate the performance gain of our proposed scheme.
