  This paper studies the problem of information freshness-aware task offloading
in an air-ground integrated multi-access edge computing system, which is
deployed by an infrastructure provider (InP). A third-party real-time
application service provider provides computing services to the subscribed
mobile users (MUs) with the limited communication and computation resources
from the InP based on a long-term business agreement. Due to the dynamic
characteristics, the interactions among the MUs are modelled by a
non-cooperative stochastic game, in which the control policies are coupled and
each MU aims to selfishly maximize its own expected long-term payoff. To
address the Nash equilibrium solutions, we propose that each MU behaves in
accordance with the local system states and conjectures, based on which the
stochastic game is transformed into a single-agent Markov decision process.
Moreover, we derive a novel online deep reinforcement learning (RL) scheme that
adopts two separate double deep Q-networks for each MU to approximate the
Q-factor and the post-decision Q-factor. Using the proposed deep RL scheme,
each MU in the system is able to make decisions without a priori statistical
knowledge of dynamics. Numerical experiments examine the potentials of the
proposed scheme in balancing the age of information and the energy consumption.
