  The use of small cell deployments in heterogeneous network (HetNet)
environments is expected to be a key feature of 4G networks and beyond, and
essential for providing higher user throughput and cell-edge coverage. However,
due to different coverage sizes of macro and pico base stations (BSs), such a
paradigm shift introduces additional requirements and challenges in dense
networks. Among these challenges is the handover performance of user equipment
(UEs), which will be impacted especially when high velocity UEs traverse
picocells. In this paper, we propose a coordination-based and context-aware
mobility management (MM) procedure for small cell networks using tools from
reinforcement learning. Here, macro and pico BSs jointly learn their long-term
traffic loads and optimal cell range expansion, and schedule their UEs based on
their velocities and historical rates (exchanged among tiers). The proposed
approach is shown to not only outperform the classical MM in terms of UE
throughput, but also to enable better fairness. In average, a gain of up to
80\% is achieved for UE throughput, while the handover failure probability is
reduced up to a factor of three by the proposed learning based MM approaches.
