  Fog radio access networks (F-RANs) are seen as potential architectures to
support services of internet of things by leveraging edge caching and edge
computing. However, current works studying resource management in F-RANs mainly
consider a static system with only one communication mode. Given network
dynamics, resource diversity, and the coupling of resource management with mode
selection, resource management in F-RANs becomes very challenging. Motivated by
the recent development of artificial intelligence, a deep reinforcement
learning (DRL) based joint mode selection and resource management approach is
proposed. Each user equipment (UE) can operate either in cloud RAN (C-RAN) mode
or in device-to-device mode, and the resource managed includes both radio
resource and computing resource. The core idea is that the network controller
makes intelligent decisions on UE communication modes and processors' on-off
states with precoding for UEs in C-RAN mode optimized subsequently, aiming at
minimizing long-term system power consumption under the dynamics of edge cache
states. By simulations, the impacts of several parameters, such as learning
rate and edge caching service capability, on system performance are
demonstrated, and meanwhile the proposal is compared with other different
schemes to show its effectiveness. Moreover, transfer learning is integrated
with DRL to accelerate learning process.
