  Mobile edge computing (MEC) networks bring computing and storage capabilities
closer to edge devices, which reduces latency and improves network performance.
However, to further reduce transmission and computation costs while satisfying
user-perceived quality of experience, a joint optimization in computing,
pushing, and caching is needed. In this paper, we formulate the joint-design
problem in MEC networks as an infinite-horizon discounted-cost Markov decision
process and solve it using a deep reinforcement learning (DRL)-based framework
that enables the dynamic orchestration of computing, pushing, and caching.
Through the deep networks embedded in the DRL structure, our framework can
implicitly predict user future requests and push or cache the appropriate
content to effectively enhance system performance. One issue we encountered
when considering three functions collectively is the curse of dimensionality
for the action space. To address it, we relaxed the discrete action space into
a continuous space and then adopted soft actor-critic learning to solve the
optimization problem, followed by utilizing a vector quantization method to
obtain the desired discrete action. Additionally, an action correction method
was proposed to compress the action space further and accelerate the
convergence. Our simulations under the setting of a general single-user,
single-server MEC network with dynamic transmission link quality demonstrate
that the proposed framework effectively decreases transmission bandwidth and
computing cost by proactively pushing data on future demand to users and
jointly optimizing the three functions. We also conduct extensive parameter
tuning analysis, which shows that our approach outperforms the baselines under
various parameter settings.
