  In recent years, over-the-air aggregation has been widely considered in
large-scale distributed learning, optimization, and sensing. In this paper, we
propose the over-the-air federated policy gradient algorithm, where all agents
simultaneously broadcast an analog signal carrying local information to a
common wireless channel, and a central controller uses the received aggregated
waveform to update the policy parameters. We investigate the effect of noise
and channel distortion on the convergence of the proposed algorithm, and
establish the complexities of communication and sampling for finding an
$\epsilon$-approximate stationary point. Finally, we present some simulation
results to show the effectiveness of the algorithm.
