  In multi-access edge computing (MEC), most existing task software caching
works focus on statically caching data at the network edge, which may hardly
preserve high reusability due to the time-varying user requests in practice. To
this end, this work considers dynamic task software caching at the MEC server
to assist users' task execution. Specifically, we formulate a joint task
software caching update (TSCU) and computation offloading (COMO) problem to
minimize users' energy consumption while guaranteeing delay constraints, where
the limited cache size and computation capability of the MEC server, as well as
the time-varying task demand of users are investigated. This problem is proved
to be non-deterministic polynomial-time hard, so we transform it into two
sub-problems according to their temporal correlations, i.e., the real-time COMO
problem and the Markov decision process-based TSCU problem. We first model the
COMO problem as a multi-user game and propose a decentralized algorithm to
address its Nash equilibrium solution. We then propose a double deep Q-network
(DDQN)-based method to solve the TSCU policy. To reduce the computation
complexity and convergence time, we provide a new design for the deep neural
network (DNN) in DDQN, named state coding and action aggregation (SCAA). In
SCAA-DNN, we introduce a dropout mechanism in the input layer to code users'
activity states. Additionally, at the output layer, we devise a two-layer
architecture to dynamically aggregate caching actions, which is able to solve
the huge state-action space problem. Simulation results show that the proposed
solution outperforms existing schemes, saving over 12% energy, and converges
with fewer training episodes.
