  Multicasting is an efficient technique for simultaneously transmitting common
messages from the base station (BS) to multiple mobile users (MUs). Multicast
scheduling over multiple channels, which aims to jointly minimize the energy
consumption of the BS and the latency of serving asynchronized requests from
the MUs, is formulated as an infinite-horizon Markov decision process (MDP)
problem with a large discrete action space, multiple time-varying constraints,
and multiple time-invariant constraints. To address these challenges, this
paper proposes a novel distribution-embedding multi-agent proximal policy
optimization (DE-MAPPO) algorithm, which consists of one modified MAPPO and one
distribution-embedding module: The former one handles the large discrete action
space and time-varying constraints by modifying the structure of the actor
networks and the training kernel of the conventional MAPPO; and the latter one
iteratively adjusts the action distribution to satisfy the time-invariant
constraints. Moreover, a performance upper bound of the considered MDP is
derived by solving a two-step optimization problem. Finally, numerical results
demonstrate that our proposed algorithm outperforms the existing ones and
achieves comparable performance to the derived benchmark.
