  To improve the quality of computation experience for mobile devices,
mobile-edge computing (MEC) is a promising paradigm by providing computing
capabilities in close proximity within a sliced radio access network (RAN),
which supports both traditional communication and MEC services. Nevertheless,
the design of computation offloading policies for a virtual MEC system remains
challenging. Specifically, whether to execute a computation task at the mobile
device or to offload it for MEC server execution should adapt to the
time-varying network dynamics. In this paper, we consider MEC for a
representative mobile user in an ultra-dense sliced RAN, where multiple base
stations (BSs) are available to be selected for computation offloading. The
problem of solving an optimal computation offloading policy is modelled as a
Markov decision process, where our objective is to maximize the long-term
utility performance whereby an offloading decision is made based on the task
queue state, the energy queue state as well as the channel qualities between MU
and BSs. To break the curse of high dimensionality in state space, we first
propose a double deep Q-network (DQN) based strategic computation offloading
algorithm to learn the optimal policy without knowing a priori knowledge of
network dynamics. Then motivated by the additive structure of the utility
function, a Q-function decomposition technique is combined with the double DQN,
which leads to novel learning algorithm for the solving of stochastic
computation offloading. Numerical experiments show that our proposed learning
algorithms achieve a significant improvement in computation offloading
performance compared with the baseline policies.
