  Network slicing is a critical technique for 5G communications that covers
radio access network (RAN), edge, transport and core slicing.The evolving
network architecture requires the orchestration of multiple network resources
such as radio and cache resources. In recent years, machine learning (ML)
techniques have been widely applied for network management. However, most
existing works do not take advantage of the knowledge transfer capability in
ML. In this paper, we propose a deep transfer reinforcement learning (DTRL)
scheme for joint radio and cache resource allocation to serve 5G RAN slicing.
We first define a hierarchical architecture for joint resource allocation. Then
we propose two DTRL algorithms: Q-value-based deep transfer reinforcement
learning (QDTRL) and action selection-based deep transfer reinforcement
learning (ADTRL). In the proposed schemes, learner agents utilize expert
agents' knowledge to improve their performance on current tasks. The proposed
algorithms are compared with both the model-free exploration bonus deep
Q-learning (EB-DQN) and the model-based priority proportional fairness and
time-to-live (PPF-TTL) algorithms. Compared with EB-DQN, our proposed
DTRL-based method presents 21.4% lower delay for Ultra Reliable Low Latency
Communications (URLLC) slice and 22.4% higher throughput for enhanced Mobile
Broad Band (eMBB) slice, while achieving significantly faster convergence than
EB-DQN. Moreover, 40.8% lower URLLC delay and 59.8% higher eMBB throughput are
observed with respect to PPF-TTL.
