  This paper targets at the problem of radio resource management for expected
long-term delay-power tradeoff in vehicular communications. At each decision
epoch, the road side unit observes the global network state, allocates channels
and schedules data packets for all vehicle user equipment-pairs (VUE-pairs).
The decision-making procedure is modelled as a discrete-time Markov decision
process (MDP). The technical challenges in solving an optimal control policy
originate from highly spatial mobility of vehicles and temporal variations in
data traffic. To simplify the decision-making process, we first decompose the
MDP into a series of per-VUE-pair MDPs. We then propose an online long
short-term memory based deep reinforcement learning algorithm to break the
curse of high dimensionality in state space faced by each per-VUE-pair MDP.
With the proposed algorithm, the optimal channel allocation and packet
scheduling decision at each epoch can be made in a decentralized way in
accordance with the partial observations of the global network state at the
VUE-pairs. Numerical simulations validate the theoretical analysis and show the
effectiveness of the proposed online learning algorithm.
