  This paper investigates the spectrum sharing problem in vehicular networks
based on multi-agent reinforcement learning, where multiple vehicle-to-vehicle
(V2V) links reuse the frequency spectrum preoccupied by
vehicle-to-infrastructure (V2I) links. Fast channel variations in high mobility
vehicular environments preclude the possibility of collecting accurate
instantaneous channel state information at the base station for centralized
resource management. In response, we model the resource sharing as a
multi-agent reinforcement learning problem, which is then solved using a
fingerprint-based deep Q-network method that is amenable to a distributed
implementation. The V2V links, each acting as an agent, collectively interact
with the communication environment, receive distinctive observations yet a
common reward, and learn to improve spectrum and power allocation through
updating Q-networks using the gained experiences. We demonstrate that with a
proper reward design and training mechanism, the multiple V2V agents
successfully learn to cooperate in a distributed way to simultaneously improve
the sum capacity of V2I links and payload delivery rate of V2V links.
