  Due to the high variability of the traffic in the radio access network (RAN),
fixed network configurations are not flexible enough to achieve optimal
performance. Our vendors provide several settings of the eNodeB to optimize the
RAN performance, such as media access control scheduler, loading balance, etc.
But the detailed mechanisms of the eNodeB configurations are usually very
complicated and not disclosed, not to mention the large key performance
indicators (KPIs) space needed to be considered. These make constructing a
simulator, offline tuning, or rule-based solutions difficult. We aim to build
an intelligent controller without strong assumption or domain knowledge about
the RAN and can run 24/7 without supervision. To achieve this goal, we first
build a closed-loop control testbed RAN in a lab environment with one eNodeB
provided by one of the largest wireless vendors and four smartphones. Next, we
build a double Q network agent trained with the live feedback of the key
performance indicators from the RAN. Our work proved the effectiveness of
applying deep reinforcement learning to improve network performance in a real
RAN network environment.
