  In this paper, we focus on a wireless-powered sensor network coordinated by a
multi-antenna access point (AP). Each node can generate sensing information and
report the latest information to the AP using the energy harvested from the
AP's signal beamforming. We aim to minimize the average age-of-information
(AoI) by adapting the nodes' transmission scheduling and the transmission
control strategies jointly. To reduce the transmission delay, an intelligent
reflecting surface (IRS) is used to enhance the channel conditions by
controlling the AP's beamforming vector and the IRS's phase shifting matrix.
Considering dynamic data arrivals at different sensing nodes, we propose a
hierarchical deep reinforcement learning (DRL) framework to for AoI
minimization in two steps. The users' transmission scheduling is firstly
determined by the outer-loop DRL approach, e.g. the DQN or PPO algorithm, and
then the inner-loop optimization is used to adapt either the uplink information
transmission or downlink energy transfer to all nodes. A simple and efficient
approximation is also proposed to reduce the inner-loop rum time overhead.
Numerical results verify that the hierarchical learning framework outperforms
typical baselines in terms of the average AoI and proportional fairness among
different nodes.
